{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 04 - Logistic Regression\n",
    "\n",
    "by [Alejandro Correa Bahnsen](http://www.albahnsen.com/) & [Iván Torroledo](http://www.ivantorroledo.com/)\n",
    "\n",
    "version 1.2, Feb 2018\n",
    "\n",
    "## Part of the class [Machine Learning for Risk Management](https://github.com/albahnsen/ML_RiskManagement)\n",
    "\n",
    "\n",
    "This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License](http://creativecommons.org/licenses/by-sa/3.0/deed.en_US). Special thanks goes to [Kevin Markham](https://github.com/justmarkham)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review: Predicting a Continuous Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type\n",
       "id                                                                        \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1\n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6\n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import zipfile\n",
    "with zipfile.ZipFile('../datasets/glass.csv.zip', 'r') as z:\n",
    "    f = z.open('glass.csv')\n",
    "    glass = pd.read_csv(f, sep=',', index_col=0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Pretend that we want to predict **ri**, and our only feature is **al**. How could we do it using machine learning?\n",
    "\n",
    "**Answer:** We could frame it as a regression problem, and use a linear regression model with **al** as the only feature and **ri** as the response.\n",
    "\n",
    "**Question:** How would we **visualize** this model?\n",
    "\n",
    "**Answer:** Create a scatter plot with **al** on the x-axis and **ri** on the y-axis, and draw the line of best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f1c4a762da0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+UXGWd5/HPt6qrOwECOEnkVwgZJjCaIInQ8mOjTkDPLmAGnEPcwV+s67KsrKhzGAXHHRTh6BlEXYUwy6JwMCODg+Ah/JwdNLAQFcYGOxkCyGRUzA+XhABJGkJ3uvq7f1RVU1V9q+qp6rp1b3W/X+f0oai6detbtzv3e5/n+9znMXcXAACNZJIOAADQHUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQXqSDqCd5syZ4wsWLEg6DADoGk888cSL7j43ZNsplTAWLFiggYGBpMMAgK5hZs+HbkuXFAAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQJDYEoaZ3Wxm283sqRqvLzezXWY2WPz5YtXrWTP7pZndG1eMAIBwPTHu+xZJqyStrrPNo+6+osZrn5H0jKQD2xwXAKAFsbUw3P0RSS+18l4zmyfpfZK+29agAAAtS7qGcaqZrTezB8xscdnz35J0qaSxRjswswvNbMDMBnbs2BFboAAw3SWZMJ6UdJS7L5F0naS7JMnMVkja7u5PhOzE3W9093537587d2580QLANJdYwnD33e4+VHx8v6Scmc2RtEzS2Wb2W0k/kHS6mX0/qTgBAAWJJQwzO9TMrPj4pGIsO939r9x9nrsvkHSepLXu/pGk4gQAFMQ2SsrMbpO0XNIcM9si6UuScpLk7jdIWinpIjMblbRX0nnu7nHFAwCYHJtK5+j+/n4fGBhIOgwA6Bpm9oS794dsm/QoKQBAlyBhAACCkDAAAEFIGACAICQMAEAQEgYAIAgJAwAQhIQBAAhCwgAABCFhAACCkDAAAEFIGACAICQMAEAQEgYAIAgJAwAQhIQBAAhCwgAABCFhAACCkDAAAEFIGKiwc2hY6ze/op1Dw0mHAiBlepIOAOmxZnCrLrtzg3KZjPaNjelr5x6vs5cekXRYAFKCFgYkFVoWl925Qa/vG9Oe4VG9vm9Ml965gZYGgHEkDEiStry8V7lM5Z9DLpPRlpf3JhQRgLQhYUCSNO9NM7VvbKziuX1jY5r3ppkJRQQgbUgYkCTNPqBPXzv3eM3IZTSrr0czchl97dzjNfuAvqRDA5ASFL0x7uylR2jZwjna8vJezXvTTJIFgAokDFSYfUAfiQJAJLqkAABBSBgAgCAkDABAEBIGACAICQPoIsz1hSQxSgroEsz1haTRwgC6AHN9IQ1IGEAXYK4vpAEJA+gCzPWFNIgtYZjZzWa23cyeqvH6cjPbZWaDxZ8vFp+fYWb/bGbrzWyjmX05rhiBbsFcX0iDOIvet0haJWl1nW0edfcVVc8NSzrd3YfMLCdpnZk94O6PxRQn0BWY6wtJiy1huPsjZraghfe5pKHi/+aKP96+yIDuxVxfSFLSNYxTi11PD5jZ4tKTZpY1s0FJ2yU96O6PJxciAEBKNmE8Kekod18i6TpJd5VecPe8uy+VNE/SSWZ2XK2dmNmFZjZgZgM7duyIPWgAmK4SSxjuvtvdh4qP75eUM7M5Vdu8IulhSWfU2c+N7t7v7v1z586NM2QAmNYSSxhmdqiZWfHxScVYdprZXDM7uPj8TEnvlfRsUnECAApiK3qb2W2SlkuaY2ZbJH1JhQK23P0GSSslXWRmo5L2SjrP3d3MDpP0PTPLqpBEbnf3e+OKEwAQJs5RUh9s8PoqFYbdVj+/QdLb44oLANCapEdJAQC6BAkDABCEhJFCrHkAII1YDyNlWPMAQFrRwkiR0DUPaIEASAItjBTZ8vJe9WSs4rnSmgel+YNogQBICi2MFHlq6y4NDecrnitf84BV1wAkiYSREjuHhnXVfU9PeP7yFYskSes3v6KN23ZNWHUtI9PGbbs7EiOA6Y0uqZQoLcH5ut5YVW3/3qx2Do1o2dVrlctkNJLPa6xqovfX9uX1X1cP6JqVdE0BiBctjJSIWoJzdGxM1z+0abwLanjU5e7qzVbWOYZH6ZoCED8SRkpELcF58WnHqDdb+SuamevRF856q/brzVY8XyqOTxWMBAPShy6pFKleglOSrn94U8U2+8bG9M6Fc/Q3//jshOdL7+l2jAQD0okWRsrMPqBPS448eHwpzupWx9fOPV4LD5kV+fxUWLqTkWBAetHCSLnqVkcpKdR6vttFFf+r70UBkAwSRhcotTZCn+9mUcX/qdTdBnQzuqSQKrW64aZaYgS6ES0MpM5U6m7bOTQ8Jb4HIJEwkFJTobuN0V6YauiSAmLAaC9MRSQMtB033b0x2qvcVLu5EtMPXVJoK7phChjthamIFgbaZjLdMJ1olXSy5cNoL0xFtDDQNq3edNeJVkkSLZ+pNNoLkGhhxGY69uPX64apdTw6URxOsgBdPtUL0O1oYcQgqX78pMf8l7phLq367us2vVjzeHRiKhCmGwHag4TRZuVXs6UT1KV3btCyhXNiPTmlpdgcNePusqvX1jwenSgOU4AG2oMuqTZLYjhl2sb8l3fDNDoenSgOd3sBejp2byKdaGG0WaevZncODeue9dtkVc+npcsl5Hh0ojjcrQXotLQcAalBC8PMDiz+9w+ifjoTYnfY9MIe3TGwWS+/OhLL1WzUVeaawa06+as/1hX3PK29+8KSVKOr1XZfzYZe3XeiONxtBei0tRyBRi2MvzezP5X0oqTfShUXsi7p6Jji6ipfvOtftPqx343///mnztdPLzu9bVezUVeZyxbO0aV3rNfo2MTt+3omnpR3Dg3r1sd/p+sf2qTebPTValxXs916dZ80ivVIm7oJw91XSJKZDbr7CZ0JqbtsemFPRbKQpNU//53OP2WBlhx58KT3X6uIfuNHT1TWMpLyFdv39WT0nfP79e5j544/d+tjz+uKezZqX94lScOjE4vP7SjW1xul1exkgu0c8ZX06LFWUaxH2oTWMH5mZu9w91/EGk0XGtz8Ss3nFx4ya9L7r3WVKZnyHtG8kLT48APHH//Pf/qVvr12U+R25Verk72abWfrpN37uvSODcpmTPkx1zUru6cGUGuYcjclPUwtoQnjdEmfMLPnJb2qQteUu/vxsUXWJZbWaEXUel5q7oq31lXm4sMP1DUrl+iS2wfHu6VyWdMXVywaH4F09QPP6PYnttbc90j+javVyVzNtnMocTtbOvv3ZvXZH64fb1lJ0l/+cH3sQ5zbie48pElowjgz1ii62MJDZun8U+dr9c8raxi1WhfNXj3Xu8osnUw2btstybX5pb266r6nlctkNDw6qpF8zd1Kkj6+bMH4CajZq9nyk/Lg5lfUk6kcp9VqX3s7Wzqvj45qX9Ux2Jd3bdy2S+8+9s1NxZWkqbA2CKaGoITh7s/HHUg3u/Kct+n8UxZocPMrWnrkwTWTRatXz/WuMmcf0Kd3HztXO4eGdeHfPVGx70ZO/aPZwZ9TrnRS9jHXcN7V12MaHvWKbVrtaw9p6dRqoUUd32jVg5ABhOA+jDZZeMishjWLyVw9N7rKjNp3PbmsafHhBzX9OeUn5ZLyZLF/X1b5MW+5r71RS6deCy3kGPRkKms8Ud8v6e6fVmNIQ+yY2kgYHRTnqJeofZsKY59LMibNyGXHi7+tnFTqnZT3783qy3+6WKe95c2TOmFFtXR2Dg1r47bduvSODRoeDZ9mpCcjZczUk8ko72O6ZuWShi2nJG+SazWGNMSOqS+2hGFmN0taIWm7ux8X8fpySWsk/ab41I/c/UozO1LSakmHShqTdKO7fzuuODspzlEvtfa96LADx7vK3rR/76SvQKNOyiV590kni5Lylk7pZJiRjQ8JLilvodU6BvW62cprMUnMAVYdSysxJDV/GaafOFsYt0hapcLJv5ZHS/d6lBmV9Jfu/qSZzZL0hJk96O5PxxRnR8U56qXWvsu7yib7eeUn5VINY0auMGFAHEM+o7rAyoVOMxIVV/lV+fBoXpmAwn2c3T6tdllygx86JbaE4e6PmNmCFt73e0m/Lz7eY2bPSDpCUmwJo1N9v+Wf046b+qJ0YkRN+Ul5/96sXh3Jx3aDXa0usP16sxrz6FpJyDGILJDn6xfu4+72abXLkhv80ClJ1zBONbP1krZJ+qy7byx/sZhw3i7p8bgC6FTf71TrY253Yqp1fKJOhn09phs+coIWH35QyzHUq8VEtZg60e3TapclN/ihU5JMGE9KOsrdh8zsLEl3STqm9KKZHSDpTkl/4e67a+3EzC6UdKEkzZ8/v6kAOtX32+7P6YbRMCExNqofLDrsQL06ktflKxbpqnufrjgZTvY+inq1mLEx1/2ffldFV16nun3qdVnWO6bc4IdOSCxhlCcBd7/fzP7WzOa4+4tmllMhWdzq7j9qsJ8bJd0oSf39/V5v22qdOgm083PS1FKpdQK79bHn9eV7n1Zv1jRaHGJbfTJrVD8YG3Odee2j6uvJaHTMdfn7Fum4Iw5q6WQYFWfpqvyzP1yvkaquqL6eQjdbuWa7fSaT1KNabyG/d27wQ9wSSxhmdqikF9zdzewkFaZa32lmJukmSc+4+zfjjKFTfb/t+pw0jYapdQK79bHn9T/uekqSNDJa2PaS2weVzWTGZ8m95L3H6usPPqeR0dr1g9JJfF++cOK+6r6n9dPLTm9rgj176RFadNiBOuu6dRopG31V63fzyeULtapqtt9GxfR2JPU0/d4xvcW24p6Z3Sbp55L+2My2mNl/MbNPmNknipuslPRUsYZxraTz3N0lLZP0UUmnm9lg8eesOGLs1Eps7fqcJFbzK1daK2PTC3si12nY9MIeffmejRPeNzpWmCG3tO1XH3i24gQdIpux8e8ZumZHyHoSCw+Zpa+vLPxu9u/Nqjdruvx9iyp+N2sGt2rZ1Wt14yO/luS68N1H66eXnR6ZBOJYwyLp3ztQEucoqQ82eH2VCsNuq59fpw7O3dCpvt92fE6So2HKu5lGRscih6AObn5FFtNvbl/etX9vVtf+5F/rrulRbuO2XcqPVbZcfMzHuwJL3UbLFs7R5e9bVPh+PRlddd/TmjWjR2cvPSLy6v76hzfpQydX1stK+9q1d6Tt3ZyTnRiSugbaJelRUqnQLX2/IaNh4jhBRHUzVXch7d03qgWz95swp5QkZW3C5hNkTBqrs837lx6u91336Pj+o9b0KLdmcKs+98PBCZMPDhcTT3m30Ug+rzEvJKXS9yvtN6T+FLWvcpNN6q2OgkpTvQtTAwmjQ9r1j7deSyWOE8TOoeHIbqYor+3LKyNNGKjaKFlItZNFLmv63L//Y33zx89FJqPyFkN5zJfduSFytt4ZuYy27Xq94SSFpaTQ6Oo+qgXSkyksZNWo1tGMZluo1D0QBxJGB4T+4w1tHUS1iOI6QWx5ea9y2cKVcz2jY9LaZ7bXnPavN2sTRiOFqp4OpOK1Youh3JaX98rrNVfkDScpLCWFRlf3US2QmbkeXf/hE3TQzFxbW3rNtIS5+xtxIGF0QLPdGq20DuI6Qcx700yN1j35vuGWn9eeBb/VZLEv7/r2j59TRONCUqEFUhoCW0q4a595QcMRn1da63zx4QfVTIBRs+3Wu7qvt8BVkidm7v5GHEgYHdBKt0azrYO4ThDrNr2ofI0b3DqlVrKQ3iiGlxJu1mzCPRRSIbGUr3V+8WnH6BsPPlexTflsu5K0fvMr4wmi3tV96HDbRtpZf2ql7kGBHI2QMDqglW6NZlsHcUwPUUpkTY6A7ai+rFXUJOpz7Rwa1uwD+vShk+frWz9+rqK+Mjya12lvebPWbXqxooh98WnH6EMnz59wLEsjx3IZaWxsTB9ZtkAXvOvolo55oxZmKyfzZuoeFMgRgoTRIa10azTbOmj3EOFa8y3NzGW0t+HJuTMsU1j1o3qJ2CifvPWXFdO+m5nkb2QMM9PLr45MaO1948HndN3af9XXP7Bk/CRaMXKs+P7/9X9/rXl/sJ8+fPJRTX2HTS/s0ed+uEEj+egW5mRO5q1OxEiBHFFiu3EPE80+oE9Ljjw4cnbVdt1AWOszWlFr4r///dET9ZU/O04zchnN7Onccqd9PaaL/uRo9fVktF8uo56MdMl7j9Xml/ZqaHhiN9SMnsIfeE/GtC/v4zfSXXL7oM689tEJtZnebOFekuqb5KRCDeaS2we16YU9euS5HbqixsixL929UZte2BP8ndYMbtV/+NYjGslHr/MRx42A1bgxEKFoYaREGiePq9XNVZr474zFh2rLy3t107pf6+71v489nj9/x5G67My3auerI7p9YIsk6asPPKtsxN2Cxx9xoDZsLUxXNlaVGApdbBMLI6+O5PXSayM1JyUcHZPOunbdeAKK3CbvOuu6dfr6ysrFq8onMiyfdPHSO9ZHDjseyRdamJ0Y7ZT2Ajm1lfQgYaRIGm8grJfISvFe+8ETdNzh/6Zr/ulXypr0er0q9STcPrBFZx9/+HiyKMl75edlpfFk0axvPvicLl+xSFfc/dSEm/6kwok86v6Oim1Gx/SZHwxWpKTzT52vK89524RJF92jj9XHly0YP9btPJnXm4hxsvWvOE7sUd1xIRdWJJl4WK0/2G7U39/vAwMDSYcxbe0cGtZDz27XFfdsjOwimqz9cll9+JT5+s6jv2m8cYtm9fXo+xecrP17szrj2xO7rSbjpvNP1Cdv+2VAcV5a/fF3jLfk7h7c2tSys7U0W1hv5qQb102jy65eW3G8ejKqmMgy6nMo4DfHzJ5w9/6gbUkYaKeof+Sh9uvN6LWR+u8rlLjjMyOXGZ8V9+7BrfrcHRuUzZj2jeblskklkGxGymUzFcemNyNVf+Vc1vTYX72n5k2d5aO4Qk+IUb+X8u9arZmTbrP7DrV+8yv6yHcf157h0ZrbVH9OXLFMZc0kDIreaKvyAn4u8K+rryejr7z/OF159nHav7f+m+JMFqUb+8pv2PvZ50/XJ/7kj5TJZJQtlkp6sxNrJn1ZU29PYdBC1Y3n4/JjmpBIM9mMvnDmW9SbzWhmLqO+noy+8YElkQMjSsv6tlIEb6aw3WyhPWrfWTM99Oz2porz1bMQ11vkqtZ3oIAfL2oYaLvSOhNnXvuoGp3iL1p+tC5459HjXSB/vWbiNpOZViTUfrmsbvjoieM39pX724c3VUxP4u7q68lUPGcZ030Xv3N8ffNv/+Q5rf757ybsqy9rcjP1VXWpnHvivKDun1aL4M0Utpv9jKh9vzqS1xX3bNRfr3mqYQto59Cwbn38d5GzEJfXVsoniaz1HdJewO92tDAQi1dH8prRM/FSO5sxzcxllMuavvL+43TZGW+dUHwtX5viC2e+ZcJU6nEYk2vx4QdOeD7qinVGrkcXn7ZwwjDohYfMGh/SfOU5b9Md/+0U5apaI5Yx3f+pd+r7F5xcsaZG6HDoyZwQP7l8ofp6Gg/drvUZ+/dmI9chqf69lQwN5xu2TtYMbtW/+5uf6JsPPlexZkrpPWcvPUI/vex0ff+Ck/Wzz79H3/jAkrrDzzu1xk2ahK4P0w60MBCLWvdw3Pepd41fhUf9I44alXXoQTPGrzL37huVmSkjRc4XJTWeTr2vJ6M/f8c83T6wpeGooFonzw+dPF8fOnl+3VZB/x/O1jc+sGRCwbp8iG2zWhnRVF6PKC0AFXXner3P+I8nztOKVetq1jRKv7eoQQ+1Wielrq+oWYjL31M+ejBk+Hkah6jHpdMFforeiE3U6J5W/5jLi76Sxu9jeOCp/6dVD21SNmPKj7muWVkYQbRx227t3juigedf0t8/vlm5noxG82MV03yEjgKa7PeIY4hn6D4nUwQuv19kxap1Qfto5vPqFbUpVDfWrgJ/M0VvWhiITTuv9KrvUSk9/tQhsyKv9Eu1iBVLjtCnTj+27n0kcX+POO6vCd3nZG78K33G+uLd7yH7aKYFVKuoXT34ANGSmMKehIFYdeJmxEaf0Y4Y0nhTZYh2FIGb3Udogq1OLvUmesRESRT4SRhAF2i1W6sdd3G3so9Otd6mszhmqG6EGgaQcu0obLajjsJ0G+k02d8LNQxgimjX1OPTuVtuquvk74X7MIAU485lpAktDCDFuHO5c8qHEde7V2g6I2FgypoKfe5JFDano1KdyMdcw3nXjOJEaMx0W4mEgSlpKk1xzUiieJXXiUpKj1mqthI1DEw5nVjWtNPaufQuKkXViUqoF1UiYWBSOjnxWSgKxWhGvWnUqRdVoksKLUtrtw+FYjSjvE4UVcOgVfcGEgZa0q77A+JAoRjNKq8TMUqqNhIGWpLExGfNoFCMZnFjYmMkDLSkG7p9OnUCmArDd4EQFL1rSGMxN02m48pmUdYMbtWyq9fqI999XMuuXqu7B7cmHRIQG1oYEdJazE2b6d7tk+Y6DhAHWhhVpuIY/jhN5/sDGL6L6YaEUYWTAEJ1Qx0HaCcSRhVOAghFHQfTTWw1DDO7WdIKSdvd/biI15dLWiPpN8WnfuTuV4a8N06M4UczpnsdB9NLnEXvWyStkrS6zjaPuvuKFt8bG04CaAbj9zFdxJYw3P0RM1vQ6fe2CycBAKiUdA3jVDNbb2YPmNniVnZgZhea2YCZDezYsaPd8QEAipJMGE9KOsrdl0i6TtJdrezE3W9093537587d25bAwQAvCGxhOHuu919qPj4fkk5M5uTVDwAgPoSSxhmdqiZWfHxScVYdiYVDwCgvjiH1d4mabmkOWa2RdKXJOUkyd1vkLRS0kVmNippr6Tz3N1rvdfdb4orVgBAY3GOkvpgg9dXqTB0tun3AgA6L+lRUgCALkHCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQUgYAIAgJAwAQBASBgAgCAkDABCEhAEACELCAAAEIWEAAIKQMAAAQWJLGGZ2s5ltN7Onary+3Mx2mdlg8eeLZa+dYWa/MrNNZvb5uGIEAISLs4Vxi6QzGmzzqLsvLf5cKUlmlpV0vaQzJS2S9EEzWxRjnACAALElDHd/RNJLLbz1JEmb3P3X7j4i6QeSzmlrcACApiVdwzjVzNab2QNmtrj43BGSNpdts6X4HAAgQT0JfvaTko5y9yEzO0vSXZKOkWQR23qtnZjZhZIulKT58+fHEScAQAm2MNx9t7sPFR/fLylnZnNUaFEcWbbpPEnb6uznRnfvd/f+uXPnxhozAExniSUMMzvUzKz4+KRiLDsl/ULSMWb2h2bWK+k8SXcnFScAoCC2Likzu03ScklzzGyLpC9JykmSu98gaaWki8xsVNJeSee5u0saNbOLJf0fSVlJN7v7xrjiBACEscI5emro7+/3gYGBpMMAgK5hZk+4e3/ItkmPkgIAdIkp1cIwsx2Snk86jqI5kl5MOogWdXPsUnfHT+zJ6ObYpcnFf5S7B40YmlIJI03MbCC0mZc23Ry71N3xE3syujl2qXPx0yUFAAhCwgAABCFhxOfGpAOYhG6OXeru+Ik9Gd0cu9Sh+KlhAACC0MIAAAQhYUxSo8WezOxjZrajbKGoC5KIM0rAIldmZtcWv9sGMzuh0zHWMpkFupJmZkea2UNm9oyZbTSzz0Rsk8pjHxh7Ko+9mc0ws38uzpC90cy+HLFNn5n9Q/G4P25mCzof6USBscd/rnF3flr8UWHqkn+TdLSkXknrJS2q2uZjklYlHWuN+N8t6QRJT9V4/SxJD6gwg/Apkh5POuYmYl8u6d6k46wR22GSTig+niXpuYi/m1Qe+8DYU3nsi8fygOLjnKTHJZ1Stc1/l3RD8fF5kv4h6bibiD32cw0tjMnp6sWevPEiV+dIWu0Fj0k62MwO60x09QXEnlru/nt3f7L4eI+kZzRxzZdUHvvA2FOpeCyHiv+bK/5UF3HPkfS94uM7JL2nNElqkgJjjx0JY3JCF3s6t9itcIeZHRnxelp1+2JWUQt0pUqxy+PtKlwxlkv9sa8Tu5TSY29mWTMblLRd0oPuXvO4u/uopF2SZnc2ymgBsUsxn2tIGJMTstjTPZIWuPvxkn6sN65eukFTi1mlTGmBriWSrlNhga5UMbMDJN0p6S/cfXf1yxFvSc2xbxB7ao+9u+fdfakK6+ycZGbHVW2S2uMeEHvs5xoSxuQ0XOzJ3Xe6+3Dxf78j6cQOxdYOTS1mlSZee4GuVDCznAon3Fvd/UcRm6T22DeKPe3HXpLc/RVJD0s6o+ql8eNuZj2SDlLKuj5rxd6Jcw0JY3IaLvZU1e98tgp9vt3ibknnF0fsnCJpl7v/PumgQljtBboSV4zrJknPuPs3a2yWymMfEntaj72ZzTWzg4uPZ0p6r6Rnqza7W9J/Kj5eKWmtFyvKSQqJvRPnmiTX9O567h652JOZXSlpwN3vlvRpMztb0qgKVyofSyzgKtZ4kav7VRits0nSa5L+czKRThQQe60FutJgmaSPSvqXYp+0JH1B0nwp9cc+JPa0HvvDJH3PzLIqJLHb3f3eqn+vN0n6OzPbpMK/1/OSC7dCSOyxn2u40xsAEIQuKQBAEBIGACAICQMAEISEAQAIQsIAAAQhYQAdZGa/TdtNbEAoEgYAIAgJA4iJmd1lZk8U1y+4MOl4gMniTm8gPh9395eKUzn8wszuTDogYDJIGEB8Pm1mf1Z8fKSkY5IMBpgsEgYQAzNbrsIEcae6+2tm9rCkGYkGBUwSNQwgHgdJermYLN6iwjKrQFcjYQDx+EdJPWa2QdJVkh5LOB5g0pitFgAQhBYGACAICQMAEISEAQAIQsIP7UcoAAAAHUlEQVQAAAQhYQAAgpAwAABBSBgAgCAkDABAkP8P738NMSWOJDsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c4ab36eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot using Pandas\n",
    "glass.plot(kind='scatter', x='al', y='ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'ri')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG+JJREFUeJzt3X2sXPWd3/HPx9dDuMBuHNZuIRdssyoKDSFAcuuAqCpDVw1JKHEJ1Rp1t02ryirtapfNhoqsIp4WNUhI223jrai3QSRNlkACuSUJWboViaBpYXP9FPACFXmA+DqqDazNGm6DH779Y85czx2fmfnNw5lz5t73SxoxPufMme+cy/y+83s8jggBANDNirIDAACMBxIGACAJCQMAkISEAQBIQsIAACQhYQAAkpAwAABJCksYtu+zvd/2c232b7R9yPau7HFr076f2n422z5bVIwAgHQrCzz3/ZK2SvpSh2Oeiohr2uy7MiJeHXpUAIC+FJYwIuJJ2+uLOn+e1atXx/r1I31LABhr27dvfzUi1qQcW2QNI8XltndL2ifp0xGxJ9sekv677ZD0nyNiW7sT2N4iaYskrV27VrOztGABQCrbL6ceW2bC2CFpXUQctv1RSTOSzs/2XRER+2z/DUl/bvuFiHgy7yRZMtkmSdPT0yyMBQAFKW2UVES8ERGHs+ePSarZXp39e1/23/2SviFpQ1lxAgDqSksYts+y7ez5hiyW12yfbvuXsu2nS/oHknJHWgEARqewJinbD0jaKGm17b2SbpNUk6SIuFfS9ZJutH1U0rykzRERtv+mpG9kuWSlpD+NiD8rKk4AQJoiR0nd0GX/VtWH3bZu/7Gki4uKCwDQH2Z6AwCSkDAAAElIGACAJCQMAEASEgYAIAkJAwCQhIQBAEhCwgAAJCFhAACSkDAAAElIGACAJCQMAEASEgYAIAkJAwCQhIQBAEhCwgAAJCFhAACSkDAAAElIGACAJCQMAEASEgYAIAkJAwCQhIQBAEhCwgAAJCFhAACSkDAAAElIGACAJCQMAEASEgYAIAkJAwCQpLCEYfs+2/ttP9dm/0bbh2zvyh63tuyfsL3T9reKihEAkG5lgee+X9JWSV/qcMxTEXFNm32/I+l5Sb885LgAAH0orIYREU9Ker2f19o+R9LHJP2XoQYFAOhb2X0Yl9vebfs7ti9s2v5Hkv6tpOMlxQUAaFFmwtghaV1EXCzp85JmJMn2NZL2R8T2lJPY3mJ71vbsgQMHiosWAJa50hJGRLwREYez549JqtleLekKSdfa/qmkr0q6yvaXO5xnW0RMR8T0mjVrRhE6ACxLpSUM22fZdvZ8QxbLaxHxmYg4JyLWS9os6YmI+I2y4gQA1BU2Ssr2A5I2Slpte6+k2yTVJCki7pV0vaQbbR+VNC9pc0REUfEAAAbjpVRGT09Px+zsbNlhAMDYsL09IqZTji17lBQAYEyQMAAASUgYAIAkJAwAQBISBgAgCQkDAJCEhAEASELCAAAkIWEAAJKQMAAASUgYAIAkJAwAQBISBgAgCQkDAJCEhAEASFLYDZQwnmZ2zumex1/UvoPzeveqSd384fdo06VTZYcFoAJIGFgws3NOn3nkWc0fOSZJmjs4r8888qwkkTQA0CSFE+55/MWFZNEwf+SY7nn8xZIiAlAlJAws2HdwvqftAJYXEgYWvHvVZE/bASwvJAwsuPnD79FkbWLRtsnahG7+8HtKighAldDpjQWNjm1GSQHIQ8LAIpsunSJBAMhFkxQAIAkJAwCQhIQBAEhCHwYwJli2BWUjYQBjgGVbUAU0SQFjgGVbUAUkDGAMsGwLqqCwhGH7Ptv7bT/XZv9G24ds78oet2bbT7X9F7Z3295j+46iYgTGBcu2oAqKrGHcL+nqLsc8FRGXZI87s22/kHRVRFws6RJJV9u+rMA4gcpj2RZUQWGd3hHxpO31fbwuJB3O/lnLHjG8yIDxw7ItqIKyR0ldbnu3pH2SPh0ReyTJ9oSk7ZL+lqQ/johnSowRqASWbUHZyuz03iFpXdb09HlJM40dEXEsIi6RdI6kDbbf1+4ktrfYnrU9e+DAgcKDBoDlqrSEERFvRMTh7Pljkmq2V7ccc1DS99ShLyQitkXEdERMr1mzpsiQAWBZKy1h2D7LtrPnG7JYXrO9xvaqbPukpF+T9EJZcQIA6grrw7D9gKSNklbb3ivpNtU7sBUR90q6XtKNto9Kmpe0OSLC9tmSvpj1Y6yQ9FBEfKuoOAEAaYocJXVDl/1bJW3N2f5DSZcWFRcAoD/M9AYAJCl7WC1asCIpgKoiYVRIyoqkJBQAZaFJqkK6rUjaSChzB+cVOpFQZnbOlRAtgOWGhFEhc21WHm1sZ4lrAGUiYVTIRH1aSq7zbvl2x4RCLQNA0UgYFXIs2q+x2G31RZqmABSNhFEhUwPc22CpNU3N7JzTFXc/ofNu+bauuPsJkiFQASSMCsm750Evlsrd1+jcB6qJhFEhmy6d0ueuu0hTqyZlte/TaLd9qdx9jc59oJqYh1Exzfc8aJ2XIdXvsvaJD07p4e1zJ21fKndf4/7VQDWRMCqs013WpteduWQn8L171WTuiLClUoMCxpWjw8iccTM9PR2zs7Nlh4EBtatZfe66i8YqKTIrH+PA9vaImE45lhoGKmcp3L86ZZkXYNyQMFBJ437/6k4d9+P8ubC8kTAwVDTD1NFxj6WIYbUYGuZPnNCug56Oe4wzEgaGpt/5E6OY1T3qmeN5kzCX0tBnLE80SRVguTbL9NMMM4rO4TI6oJdCxz3QimG1Q1bmkNCyE9UVdz+RO39i1WRNp79jZW5c7V4ztWpS37/lqkLjGuZ7AOOql2G1NEkNWVnLWlSh/yCvGaa2wnrz7aNt4xpF5zAd0MBwkDCGrKzCqQrrL7WuhTW1alJnnLpSR44trsU2xzWKzuFx7oBm1V5UCX0YQzbKZS1mds7p9kf36OD8kbbH9JqoBm3Wap0/cd4t3+4Y180ffk9uE94wO4dH8R5FYPIfqqZjwrD9yxHxhu0z8/ZHxOvFhDV+PjvzrB545me5N0EatHDKK8Ql6eav7daR4537oN69anLh9XMH5zVh61iEpnKSQREFVLcEOorO4XHtgGbyH6qmWw3jT23/Q0mvSvqppOZ1tUPSrxYU11j57Myz+vLTr+TuyyuYe9GuED+1tqJrspisTWj9r0zqdx/ctXDHvkZCy0sGgxRQ7WomKb/u82Z1D6sDv/U8//7XLxmbwpa+F1RNx4QREddIku1dEfGB0YQ0fh545me52yfsgUfhtCvEW7e1mlo1qdNOWaHv/6h9JbA1GfRbQKXUTHop/IdV05nZOaebv757oQ9l7uC8bv767p7PUxZW7UXVpPZh/C/bfyciflBoNGOq3b2487b3+su5n1+TE3ZuQdPt/P0WUN1qJr2uCzWsmo4stf4JjhwL3fHNPWORMMa17wVLV2rCuErSv7L9sqQ3VW+aioh4f2GRjZFGv0De9mb9/HJuV4ivmqzpzV8czW2WapfA2p2/IbWAyusTydNv08mwajpqcxn+6q32gwSqZFz7XrB0pSaMjxQaxZi74UPn5vZh3PChcxf9u59fzu0K8duvvVCSFo2SWmGpS7dG7vkbUgqo1kK5U3Lqt+mkU02nUw0t7/r2ouyJj+1iYHIhqiIpYUTEy0UHMs7u2nSRJC2MkpqwdcOHzl3Y3tDPL+duhXjKENZ2TqutOKlA7NZ8lFooD9J0kpckrXqNrLkDv7WGllqjWTVZO2lbFYawDhJDFZIdlr7C5mHYvk/SNZL2R8T7cvZvlPTfJP0k2/RIRNxp+1xJX5J0lqTjkrZFxH8oKs5huWvTRScliFb99hGk9gG0O3+e2oT1767rvUUxpVAedGRYc5KcOzhfb//M9rXWZ5praCmfv7bCC7Wz5kJ2RU7T2qiHsPbbd1OFZIflociZ3vdLurrLMU9FxCXZ485s21FJvxcRf1vSZZL+je33FhjnyBS9gmne+Rss6fRTJhZmYN9z/cV9FSbdkltjfaZBC6pNl07p+7dcpalVk+26IhY0TwLMW5rkXafVTnzuf1z/3K1LqaT0wxQ967rfvpsqzPLH8lBYDSMinrS9vo/X/VzSz7Pnf237eUlTkv5yqAE2GUV1vvEe80eOdZw8N4hRdJLmNRc1DJL82v0NUmo0/UwCTG1aa5x7FL/i+62BMl8Do1L20iCX294taZ+kT0fEnuadWcK5VNIzRQVQxvLaxyIWCtdhJ6aib23a2lw0jOTX6W/QrZkpZRJgnpTCtPnco5h13e8wWuZrYFTKTBg7JK2LiMO2PyppRtL5jZ22z5D0sKSbIuKNdiexvUXSFklau3Ztz0GMoiAY5ntUoXOzW6GcEmNK/8HvPbRbxyIW9WFIWvj3IEmqWyKasBctSd/Lr/h+/0adakidzsl8DYxKaQmjOQlExGO2/5Pt1RHxqu2a6sniKxHxSJfzbJO0TarfD6PXOMZpee0qdW62K8DyZld/6qFduuObe3TwrSN652RNbx89preOHF84V7eJj6H+k0QvS5Y0Ox6x8B4zO+dyk5p08q/4Qf9G7ZZJ6XRO5mtgVEpLGLbPkvR/IyJsb1C9A/4125b0BUnPR8QfFh3HKKrzw3qPqixG16kAu+Obe05azvx4nJgs12ll3U4ayaKXOQkzO+cWLdA4d3BeN39t8dIgjVpMq9a+i9RFJYv4G6Wcs+imSEAqdljtA5I2Slpte6+k2yTVJCki7pV0vaQbbR+VNC9pc5Y8/q6k35T0rO1d2el+PyIeKyLOcVpeu6zOzdZl1PMmCDYKsCJnUe87ON9Tc8/tj+45aSb8keOh2x/ds6iAzZvzceUFayS17xxvbbJqjrFd7P0aZOY7tQ4MU5GjpG7osn+rpK052/+nFq+KW6hRLa89+/Lriyb2feKDvf8iTKmpDLuQaP2VLrWfTZ46B2QQNz24a+F5Xo2hWbvazMH5I4uu06m1xaPLQ9LD2+c0ve7MtoVya5NVp/4YabAaaz811Co1X2Lp4I57OjHm/yd3f2wocwhazeyc08Pb5xYKkmMRenj7XM/j+LvN4yjiNq33PP5i12XUG4rO8nlRNGoMrbp95ubrNN/Un9LQqDF1u1tfynyOQWus/czfYW4GikDCGIHUL2+3iWF5t0BtbhYpopDopSml5xEHQ9Jak5jZOaffbaqJtFphJc3B2Hdwvmth3anJKu9v1I9uf/d2sfeyHUhR9jyMZSHly5vahNCpc7OIQmLVabWxWN21eQXd1mG4rVIXaAzVE8InPjil775wILeZr1OT1U/u/lhPn6GTXju1mZuBIpAwRiDlyzuM0TXDLiRmds7p8P872tdrR6052XbLB+/qIQnOHZzXw9vnTvpF3xgI0O69er3mnfqe+umX6nWgBR3kSEHCGIGUL+8wagfDHvHVS/9F2XpZ1vxQjzWm+SPHdNODuzT78uu6a9NFuQMBmvVyzes1yx8u6kdprl1K6qvzupfBHHSQIxUJYwRSvrzDqB0Me8TXUm3vPrmLO82Xn35F39gxpzff7pycViT2/ndKPM19T/3WPFObsaoyvwfVR8IYkW5f3mHVDoY5gavT8hnd+gmWqm7JonHMpx7cpU89tEvHQ4vuj5IyBLehU8Ku4koERaCprFpIGBVRxeUd2iWxz11Xv+9Ho5N5qRlGMjyuEyc5FqEvP/2KfnLgsHa8cijpboXSidrlsPql2hW+w6jdFlGw5zWV3fy13QvLzKSsU1aF79FS4ujh/s9VNz09HbOzs2WHsaSkfPn+yZ/8b33/R6+XFOHS9Ue/fomkk2ei11ZYZ5y6smOh2eqk+51rcfIf5D06nXuQgvqKu5/o+oOk9X2KimUps709IqaTjiVhYBiah7WOu6lVk7rygjW592kfldNqK/SXf/ARSYuT9jsna3rz7aOL1utKKRDbFb6N9bkGeY9OBfsgKwqfd8u3k2p6zWuMdfucOFkvCYMmKQxFo+8k5VdhlbXep+QrT78y8r6a1tvnNvdLXXH3EydNVEzpoO7WTzHIe3Tq6+g24qr5h0brvVVSbznc/P5V7o9ZCpjpjaHqdJvYTiZrE3rXabUCIupN86/ouzZd1FM7fmN292Qt/Ws1WZvQb1y2dtEs7k63z+2nQGwsz54n7/O1O9fcwfncFQi6XaN2qw00L6sinejTaSSZKy9Yk/T/UvP7d1vKZSkp+pbBeahhYKgaBd1NHZbmaNX4RSmd3JY+SlOrJpNXn23V2mTz2Zln2zZpTdg6HtFXh2yvHdS9Ls/e6T2k/BpDt3uLSPnXsdNtcuePHNN3Xzigz113UdemsubPsFxuJlXW3BkSBoZu06VTbfsznA1B6na/7UYn+5UXrNHD2+cKTyK9Fp6rJms6/R0r2w4GuGvTRZped+bQO2B7LRB7XZ693Xs0y7sXR+O92iWaXmoyzftbh4l3G4RRxdGGRShr7gwJA4XoNCS32wzl1v3T685c9CvTrt+QKW/46wpLp65cseiOfq3edVpNH3v/2W3Xh0r5HLdfe2HSxDlpuIVXr+dMWZ6903u0SwCt52383dqNUuq1JtPYnxfbsCYsjrOy+mpIGCjEMAvLTgVAt1+cg47JH/RzDKPwyvsMqSN++p1j0W0QQ7vX93K9OtVklmIz0jCVtbgkw2qBCht0XkHZr085f7tRUku9ljCIYf5dGFYLLBGDtlUPo4Y0yOtTzk9i6F1ZfTXUMIAKazd5zdJQ77eB5YsaBrBEcCOk0aBpLA0T97BklTGxadj6uZ83etNtAuE4/n9TFBIGlqTmQiA0vl/+fu7njd50m0CYN0t9uaJJCkvSUropEB3DxUqZQIg6Egb6VuX7DrAIHVL1M4FwuaJJCn2pepPPclqEDoPptGAm/UWLkTDQl05NPlVAZzFSNfcTSfV1tiT6i/LQJIW+VL3JZ1QTm6rcLId09BOlIWHkoBDobhzmBxRdCJS1xDRQFpqkWlS9bb4qaPKpfrMcMGwkjBYUAmmYH1D9Zjlg2GiSakEhkG65t/uOQ7McMEyF1TBs32d7v+3n2uzfaPuQ7V3Z49bU1xaJ4ZhIRbMclpsim6Tul3R1l2OeiohLssedPb62EBQCSEWzHJabwpqkIuJJ2+tH/dpBLZd7AmM4lnuzHJaXsvswLre9W9I+SZ+OiD29nsD2FklbJGnt2rVDCYpCAABOVuYoqR2S1kXExZI+L2mmn5NExLaImI6I6TVr1gw1QADACaUljIh4IyIOZ88fk1SzvbqseAAAnZWWMGyfZdcXbbG9IYvltbLiAQB0Vlgfhu0HJG2UtNr2Xkm3SapJUkTcK+l6STfaPippXtLmyG4wnvfaiPhCUbECALorcpTUDV32b5W0tZ/XAgBGj6VBAABJSBgAgCQkDABAEhIGACAJCQMAkISEAQBIQsIAACQhYQAAkpAwAABJSBgAgCQkDABAEhIGACAJCQMAkISEAQBIQsIAACQhYQAAkpAwAABJSBgAgCQkDABAEhIGACAJCQMAkISEAQBIQsIAACQhYQAAkpAwAABJSBgAgCQkDABAEhIGACAJCQMAkISEAQBIUljCsH2f7f22n2uzf6PtQ7Z3ZY9bm/ZdbftF2y/ZvqWoGAEA6YqsYdwv6eouxzwVEZdkjzslyfaEpD+W9BFJ75V0g+33FhgnACBBYQkjIp6U9HofL90g6aWI+HFEvC3pq5I+PtTgAAA9K7sP43Lbu21/x/aF2bYpST9rOmZvtg0AUKKVJb73DknrIuKw7Y9KmpF0viTnHBvtTmJ7i6QtkrR27doi4gQAqMQaRkS8ERGHs+ePSarZXq16jeLcpkPPkbSvw3m2RcR0REyvWbOm0JgBYDkrLWHYPsu2s+cbslhek/QDSefbPs/2KZI2S3q0rDgBAHWFNUnZfkDSRkmrbe+VdJukmiRFxL2Srpd0o+2jkuYlbY6IkHTU9m9JelzShKT7ImJPUXECANK4XkYvDdPT0zE7O1t2GAAwNmxvj4jplGPLHiUFABgTJAwAQBISBgAgCQkDAJCEhAEASELCAAAkIWEAAJKQMAAASUgYAIAkJAwAQBISBgAgCQkDAJCEhAEASELCAAAkIWEAAJIsqfth2D4g6eWy48islvRq2UEMYJzjJ/ZyEHs5Bo19XUQk3d96SSWMKrE9m3pTkioa5/iJvRzEXo5Rxk6TFAAgCQkDAJCEhFGcbWUHMKBxjp/Yy0Hs5RhZ7PRhAACSUMMAACQhYQzI9tW2X7T9ku1bcvZ/0vYB27uyx78sI848tu+zvd/2c2322/Z/zD7bD21/YNQxtpMQ+0bbh5qu+62jjrEd2+fa/q7t523vsf07OcdU8tonxl7Ja2/7VNt/YXt3FvsdOce8w/aD2XV/xvb60Ud6ssTYiy9rIoJHnw9JE5J+JOlXJZ0iabek97Yc80lJW8uOtU38f0/SByQ912b/RyV9R5IlXSbpmbJj7iH2jZK+VXacbWI7W9IHsue/JOn/5Px/U8lrnxh7Ja99di3PyJ7XJD0j6bKWY/61pHuz55slPVh23D3EXnhZQw1jMBskvRQRP46ItyV9VdLHS44pWUQ8Ken1Dod8XNKXou5pSatsnz2a6DpLiL2yIuLnEbEje/7Xkp6XNNVyWCWvfWLslZRdy8PZP2vZo7UT9+OSvpg9/7qkv2/bIwqxrcTYC0fCGMyUpJ81/Xuv8r88n8iaFb5u+9zRhDYUqZ+vqi7PqvDfsX1h2cHkyZo8LlX9F2Ozyl/7DrFLFb32tids75K0X9KfR0Tb6x4RRyUdkvQro40yX0LsUsFlDQljMHm/PFqz/jclrY+I90v6Hzrx62UcpHy+qtqh+pIHF0v6vKSZkuM5ie0zJD0s6aaIeKN1d85LKnPtu8Re2WsfEcci4hJJ50jaYPt9LYdU9ronxF54WUPCGMxeSc1Z/BxJ+5oPiIjXIuIX2T//RNIHRxTbMHT9fFUVEW80qvAR8Zikmu3VJYe1wHZN9QL3KxHxSM4hlb323WKv+rWXpIg4KOl7kq5u2bVw3W2vlPROVazps13soyhrSBiD+YGk822fZ/sU1TvJHm0+oKXd+VrV23zHxaOS/mk2YucySYci4udlB5XC9lmNtmfbG1T/f/21cqOqy+L6gqTnI+IP2xxWyWufEntVr73tNbZXZc8nJf2apBdaDntU0j/Lnl8v6YnIepTLlBL7KMqalcM+4XISEUdt/5akx1UfMXVfROyxfaek2Yh4VNJv275W0lHVf6l8srSAW9h+QPURLatt75V0m+qdaYqIeyU9pvponZckvSXpn5cT6ckSYr9e0o22j0qal7S5Cl/8zBWSflPSs1mbtCT9vqS1UuWvfUrsVb32Z0v6ou0J1ZPYQxHxrZbv6xck/VfbL6n+fd1cXriLpMReeFnDTG8AQBKapAAASUgYAIAkJAwAQBISBgAgCQkDAJCEhAGMkO2fVm0SG5CKhAEASELCAApie8b29uz+BVvKjgcYFDO9geL8i4h4PVvK4Qe2Hy47IGAQJAygOL9t+x9lz8+VdH6ZwQCDImEABbC9UfUF4i6PiLdsf0/SqaUGBQyIPgygGO+U9FdZsrhA9dusAmONhAEU488krbT9Q0l/IOnpkuMBBsZqtQCAJNQwAABJSBgAgCQkDABAEhIGACAJCQMAkISEAQBIQsIAACQhYQAAkvx/npK856kLz3oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c4a6f9240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# equivalent scatter plot using Matplotlib\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit a linear regression model\n",
    "from sklearn.linear_model import LinearRegression\n",
    "linreg = LinearRegression()\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.ri\n",
    "linreg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  \n",
       "id             \n",
       "22   1.521227  \n",
       "185  1.521103  \n",
       "40   1.520781  \n",
       "39   1.520781  \n",
       "51   1.520682  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make predictions for all values of X\n",
    "glass['ri_pred'] = linreg.predict(X)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'ri')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XtwXOWZ5/HvI7ltSb7JsuSLhG9hGUiAYEBjoJjacshkIQkJHsLuQO1kN5PaopbZqUl2JkyRVBYIQy3spmpmMiFbLFkokp2E3Ei8JJCQZAmDww4EXzHmVkCAWLItWbJsy5ZtXZ79o4/kVqtP9yupT5+W/PtUdbnd59JPt93vc97rMXdHRESklJq0AxARkZlBCUNERIIoYYiISBAlDBERCaKEISIiQZQwREQkiBKGiIgESSxhmNlDZtZlZi/FbN9oZofNbGf0uD1n29tmtjt6fWtSMYqISLg5CZ77YeA+4JtF9tni7tfGbPuAux8se1QiIjIliSUMd3/GzNYmdf5Cmpubfe3air6liMiMtm3btoPu3hKyb5I1jBBXmNkuoBP4nLvviV534Odm5sD/dPcH4k5gZjcDNwOsXr2arVvVgiUiEsrM3gndN82EsR1Y4+79ZvYRYDNwTrTtSnfvNLNlwC/M7FV3f6bQSaJk8gBAe3u7FsYSEUlIaqOk3P2Iu/dHz58AMmbWHP29M/qzC/gRsCGtOEVEJCu1hGFmK8zMoucbolh6zGy+mS2MXp8P/Cug4EgrERGpnMSapMzsEWAj0Gxme4E7gAyAu98P3ADcYmZDwABwo7u7mS0HfhTlkjnAt939Z0nFKSIiYZIcJXVTie33kR12m//6W8BFScUlIiJTo5neIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJIgShoiIBFHCEBGRIEoYIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJIgShoiIBFHCEBGRIEoYIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJIgShoiIBFHCEBGRIEoYIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJEhiCcPMHjKzLjN7KWb7RjM7bGY7o8ftedtrzWyHmf0kqRhFRCTcnATP/TBwH/DNIvtscfdrY7Z9BngFWFTmuEREZAoSq2G4+zNA71SONbOzgI8C/6usQYmIyJSl3YdxhZntMrOfmtn5Oa//PfDXwEhKcYmISJ40E8Z2YI27XwR8FdgMYGbXAl3uvi3kJGZ2s5ltNbOt3d3dyUUrInKGSy1huPsRd++Pnj8BZMysGbgS+LiZvQ18B7jKzP6xyHkecPd2d29vaWmpROgiImek1BKGma0wM4ueb4hi6XH3z7v7We6+FrgReMrd/yStOEVEJCuxUVJm9giwEWg2s73AHUAGwN3vB24AbjGzIWAAuNHdPal4RERkemw2ldHt7e2+devWtMMQEZkxzGybu7eH7Jv2KCkREZkhlDBERCSIEoaIiARRwhARkSBKGCIiEkQJQ0REgihhiIhIECUMEREJooQhIiJBlDBERCSIEoaIiARRwhARkSBKGCIiEkQJQ0REgihhiIhIkMRuoCQz0+YdHXz5ydfo7BugtbGeW68+l00Xt6UdlohUASUMGbN5Rwef/+FuBgaHAejoG+DzP9wNoKQhImqSktO+/ORrY8li1MDgMF9+8rWUIhKRaqKEIWM6+wYm9bqInFmUMGRMa2P9pF4XkTOLEoaMufXqc6nP1I57rT5Ty61Xn5tSRCJSTdTpLWNGO7Y1SkpEClHCkHE2XdymBCEiBalJSkREgihhiIhIECUMEREJoj4MkRlCy7ZI2pQwRGYALdsi1UBNUiIzgJZtkWqghCEyA2jZFqkGiSUMM3vIzLrM7KWY7RvN7LCZ7Ywet0ev15nZb8xsl5ntMbMvJRWjyEyhZVukGiRZw3gYuKbEPlvcfX30uCt67SRwlbtfBKwHrjGzyxOMU6TqadkWqQaJdXq7+zNmtnYKxznQH/01Ez28fJGJzDxatkWqQdqjpK4ws11AJ/A5d98DYGa1wDbgXwBfc/fnU4xRpCpo2RZJW5qd3tuBNVHT01eBzaMb3H3Y3dcDZwEbzOyCuJOY2c1mttXMtnZ3dycetIjImSq1hOHuR9y9P3r+BJAxs+a8ffqApynSF+LuD7h7u7u3t7S0JBmyiMgZLbWEYWYrzMyi5xuiWHrMrMXMGqPX64E/BF5NK04REclKrA/DzB4BNgLNZrYXuINsBzbufj9wA3CLmQ0BA8CN7u5mthL4RtSPUQN8z91/klScIiISJslRUjeV2H4fcF+B118ELk4qLhERmRrN9BYRkSBpD6uVPFqRVESqlRJGFQlZkVQJRUTSoiapKlJqRdLRhNLRN4BzOqFs3tGRQrQicqZRwqgiHTErj46+riWuRSRNShhVpDY7LaWgdbc9XjShqJYhIklTwqgiwx6/xmKp1RfVNCUiSVPCqCJt07i3wWxrmtq8o4Mr732Kdbc9zpX3PqVkKFIFlDCqSKF7HkzGbLn7mjr3RaqTEkYV2XRxG/dcfyFtjfUY8X0aca/PlruvqXNfpDppHkaVyb3nQf68DMjeZe0Tl7bx6LaOCa/Plruv6f7VItVJCaOKFbvLWvuaplk7ga+1sb7giLDZUoMSmanMi4zMmWna29t969ataYch0xRXs7rn+gtnVFLUrHyZCcxsm7u3h+yrGoZUndlw/+qQZV5EZholDKlKM/3+1cU67mfy55IzmxKGlJWaYbLUcS+zkYbVStlo/sRpcR306riXmUwJQ8pmqvMnKjGru9IzxwtNwpxNQ5/lzKQmqQScqc0yU2mGqUTncBod0LOh414kn4bVllmaQ0LTTlRX3vtUwfkTjfUZ5s+bUzCuuGPaGut59rarEo2rnO8hMlNNZlitmqTKLK1lLaqh/6BQM0ymxjh2aig2rkp0DqsDWqQ8lDDKLK3CqRrWX8pfC6utsZ4FdXMYHB5fi82NqxKdwzO5A1qr9ko1UR9GmVVyWYvNOzq487E99A0Mxu4z2UQ13Wat/PkT6257vGhct159bsEmvHJ2DlfiPZKgyX9SbYomDDNb5O5HzKyp0HZ3700mrJnni5t388jzvyt4E6TpFk6FCnGAW7+/i8GR4n1QrY31Y8d39A1Qa8awO20FkkESBVSpBFqJzuGZ2gGtyX9SbUrVML5tZh8DDgJvA7nrajvwnoTimlG+uHk3//jcuwW3FSqYJyOuEK/L1JRMFvWZWtYurec/f3fn2B37RhNaoWQwnQIqrmYScnVfaFZ3uTrw88/zd3+8fsYUtup7kWpTNGG4+7UAZrbT3S+pTEgzzyPP/67g67Vm0x6FE1eI57+Wr62xnoa5NTz7ZnwlMD8ZTLWACqmZTKbwL1dNZ/OODm79wa6xPpSOvgFu/cGuSZ8nLVq1V6pNaB/G/zOz33f3FxKNZoaKuxd3odcne+U8lavJWrOCBU2p80+1gCpVM5nsulDlqulgkP9PMDjsfOnHe2ZEwpipfS8ye4UmjKuA/2hm7wDHyDZNubu/P7HIZpDRfoFCr+eaypVzXCHeWJ/h2Mmhgs1ScQks7vyjQguoQn0ihUy16aRcNR1ivoZDx+MHCVSTmdr3IrNXaML4cKJRpO34cbjzTmhrg9bW03+uXAnz5pU8/KbLVhXsw7jpslXj/j6VK+e4QvzOj58PMG6UVI1BiW6NgucfFVJA5RfKxZLTVJtOitV0itXQCn2/k5H2xMe4GDS5UKpFUMJw93eSDiRVBw7AV74Cp05N3NbcPD6J5D6P/rz7Y+8DGBslVWvGTZet4u5NF4471VSunEsV4iFDWOM0ZGomFIilmo9CC+XpNJ0USpJGtkaW24GfX0MLrdE01mcmvFYNQ1inE0M1JDuZ/RKbh2FmDwHXAl3ufkGB7RuB/wP8Nnrph+5+l5mtAr4JrABGgAfc/StJxQnAunVw4gT09kJHB3R2Zh+jz0f/3LkT9u+f2DBeW8vdK1dyd24yebkVDm8fl1haF9fRcfjEhLcvdSUe2gcQd2VeSKbW+K/XT75FMaRQnu7IsNwk2dE3kG3/jLbl12dya2ghnz9TY2O1s9xCtqZA01qlh7BOte+mGpKdnBmSnLj3MHAf2cI/zpbRkVg5hoC/cvftZrYQ2GZmv3D3lxOKM8sMli7NPt5fpCAdGsrWSOISy+uvw9NPw6FDEw79p7p6OuqXsH9BE/sXLOXAgiZ6FzfzwQ9eDFsyp2sw9VNryil0ZT728YCGubUcPzU8rSvQUoVyudZnGk2ScetA5So2CTBTYyyom0Pf8cFxnzu0aS03QSZ9FT/VvhvN15BKSSxhuPszZrZ2CsftA/ZFz4+a2StAG5BYwphUQTBnTrbG0Fbih3j8OOzbN5ZMdr/wCi/95mXm93Sx4mgPF+17nZX9PcwbOgW/zDt2yZKiTWC0tsLy5dlYclSik7RYUppOM1Tcv0FIjWYqkwBDm9ZGz12Jq/ipjlLTfA2plLSXBrnCzHYBncDn3H1P7sYo4VwMPJ9UAIkVBA0NcPbZcPbZ2ffY08jAlRvGNtdnarnnjy5g07r58U1gnZ2wZ0+2GWw4r3CrqckmjbzEsqm1NRt36+rsa00FJ+lPWX5zUbGZ46GK/RuUqtGETAIsJKQwzT13Ja7ipzqMVvM1pFLSTBjbgTXu3m9mHwE2A+eMbjSzBcCjwGfd/UjcSczsZuBmgNWrV086iEoUBLHv8fPX2XTbVdkaxfnnx59geBi6uqCzk+e27GbLP+1iXtd+1p3qY8PcEyx/+2149lno6Zl47Lx5JTvtaW2F+fODP0+pQjmkxhbSf/BX39vFsPu4Pgxg7O/TSVKlElGt2bgl6SdzFT/VpqtiNaRi59R8DamU1BJGbhJw9yfM7H+YWbO7HzSzDNlk8S13/2GJ8zwAPADZ+2FMNo4Zsbx2bS2sXMnm/SN8vqebgXNbICoLxt1r48SJbDNYsU77xx+HY8cmvsfixaUTy4oVkMmOMIorwArNrv7L7+3kSz/eQ9/xQRbXZzg1NMzxwZGxty418dGZepKYzJIluUbcx95j846OgkkNJl7FT7fGGrdMSrFzar6GVEpqCcPMVgAH3N3NbAPZpdZ7zMyAB4FX3P1vk46jEtX5cr1HydpQXV12xNe6dfEncYejR4uPBnv66eyfQ0PjjzWDZcvoW9LC4uEG/lNDEwcWNLF/4VJ+uquZxdddzj/s6GOIerDTK+eP+OnJcsVW1i1mNFlMpkN9846OcQs0dvQNcOv3xy8NMlqLyZffdxG6qGQSNdaQc052Nr3IVCQ5rPYRYCPQbGZ7gTuADIC73w/cANxiZkPAAHBjlDz+APgksNvMdkan+4K7P5FEnDNpee2y1IbMYNGi7OO9743fb2QEDh6Ejg7+ectufvHL7Szq7WJZfy8r+3tYdvQgF3S8TsvxvtPHfAc+AJyqmUPXgmwyObCgiQMLl3JgwVL2L8yODOuKRokdm9cQHnf0OSfT3HPnY3smzIQfHHHufGzPuAK20JyPD5zXAsR3juc3WeXGGBf7VE1n5rtqHVJOSY6SuqnE9vvIDrvNf/3XjF8VN1GVWl576zu94yb2feLSyV8RhtRUylZI1NTAsmVs7hjk1v3NDL7vQwV3ywwP0nLsECuO9rCsv5fl/b3R8x5W9Pfwewff5Q/e3smiU8cnHNs/t54D0fDi04lldMjxUroWNtE1v4lTc05PtPvsd3eOPS9UY8gVV5vpGxgc9z3VZcbfR8yBR7d10L6mKbZQzm+yKtYfA9OrsU6lhqq5GZKEtEdJVYWkq/Obd3Tw6LaOsYJk2H2sQJrM+5aqqSRRSHz5ydeKLqM+WJuhc9EyOhctm9A5navh1MC4ZHL6eS8r+nto73iFZf09zBsemnBsT/2isWRyIEomBxaensvy1W8dYdNF/yab5CKl7kyX+z0N5PSnjBpt8ilVWIfM55hujXUqNVTNzZAkKGFUQOiPt1TtoFRtKIlCYjJNKcVGHByfW89vm9r4bVORONxZMnCE5VFNZfnRHpZHNZXR2sv5B96i+VgfNfnv9pVPZjvl29robFjCkcO1/NmCiU1hR+bNp6bGguZgdPYN8Hd/vL5oYV2syWrEvSw11qnUgjU3Q5KghFEBIT/e0NpBsdpQEoVEY0Omcqu7mnGoYTGHGhbz6rL4jvvakWFa+g+xPKqpLO/v4U9WZXhz52ssONDN8v6Xue5oD4tPThwNdjwzb3xNJbcpLOpr6Zq/hJOZeTjZQvoTl7bxq1e7CxbWxZqsfnvvR8vytcDka8GamyFJUMKogJAfbzlqB+UuJDbv6KD/xMQmorQN19Syf1Ez+xc1j732g0wtAy1Xj9uvbvDEhNrK8v5eVg8corHvIBfte50V/T3UDU1cdPJQ3cKxWkn34mY+2f5ezr30vOzw4uF9/Ozn+/nCs12xtarJfufFapdT6ZeabDOWOsglhBJGBYT8eMtROyj3iK9S/RfVpFCz0IlMHe8saeWdJa3jXq8hu6olAO4sOnmMFUcP5iWX3qgprIdz3nyXlhf/Lzx4uq/jGuBDVkP3/MaoT+V0533v4hauveZS2L07O4dlyZLs6LQCsjXLF8f1o+TWLoEp9UtNphlLHeQSSgmjAkJ+vOWoHZR7xNdsbe8e18VtxpG6BRypW8DrLWtjj6kZGWbp8cOsO3GIRYcOZmsrOYllVd9+2ve+TNNANB/1JzkH19WdngyZMxHyhcF6vr+nn+UNS9i/cCknMnVjh4zWLkef5wqteYY2Y6mDXEIpYVRIqR9vuWoH5RzxVWz5jGIjomajkZpauhc00b2gCZrPjt1v3tApWvp7aY066pcdzc5b2VB3govsGP3PvUDtvseoHzzB7wPfyjn2yLz5YyO/uqJRYF05fSsHFjTRPX8JQ7VzqmslggSpqay6KGFUiWpc3iEuid1zffbGUKMLEM4200mGJ+fMZW/jCvY2rpiw7cqzm9j+7mEGTg2x8NRxlh3N1k7GOu+Pnh5u/J53drP8WC9zRsZf+Y9g9MxfTM/iFthzf/zaYM3NE5rB4grfctRukyjYCzWV3fr9XWPLzISsU1YNv6PZxHwS93+udu3t7b5169a0w5hVQn58//br/8yzb/amFOHs9ff/+v3MO9TD17/7axb3HWRF1HG/sr+X1uOHaDp8kNZjvSw51jfx4Llzs7cYjpLIm5lF/OgA7G1YMlZbObykhf9y02XAxNnucfcRKWTCvdTJW+NsikLuhZL/PknFMpuZ2TZ3bw/aVwlDymE0scyGGkdbYz0fOK+l4H3aK6UhU8PLf/NhYHzSXlyf4dipobHFHQEW1Yzw5SuXcXXTSOwy+cd++y7zT06cbX9sXgPz166ie1Ez2wbreXveYg4vWcbe+kY6GrJDjrsWLGFOXV1soVusYJ/OisLrbns8qKaXu8ZYXCzlurHXbDSZhKEmKSmLydwhr5qN9huNFnDfeu7divfV5N8+N7df6sp7n5qw5MmRkRru2n2Mq4sUiBfc9jgNJ4+PjQQbXbpl+dFePr1uLi2dnVzT/UY2wRS4t/3BhsUcerAZ1v/ehCawJa++ycmFS+lpWIzb+GVWSo24yr3QyL+3Sugth3P7Wqq5P2Y2UMKQsiq1bHic+kwtdZmayk0SjJF7FX33pgv51avdwQlwdHZ3Xaam4HIjhdRnaotODMw3lQJxdHn2Y/MaeGteA28tPWtsW1tjPZ/OTTTuXPKZR6JRYL1jS7eMjgob3vUGa1/YRl1P99i97UcHhA3W1NI1Pzt35UBep/2v9u1m019+NJtkFi0Cs9hlVUaTzCcubePRbR0l/y/l9rWcSRMW0+irUcKQshr9D5u7SGApo1eUMLEtvZLaGuuDV5/Nl99O/sXNu2ObtKazbMhkC8TJLs+OGfWtK3ilbzGvLHtPwXPWZ2q592PncV3rHOjo4Plf7+bnv9xBU1/3WOf9e3o6uOKdF8fPtn/wr7N/NjRAWxurhxq4t66R/QuXjq1gfGBhdhZ+14ImfvVqN/dcf2HR5rj8z3Cm3EwqrbkzShhSdpsubovtz7BoCFKp+22PXjV94LyWoKvM6YorVOIK6Mb6DPPnzYm9urt704W0r2kqewfsZAvEyS7PHvceuQYGh/nvT73FdbddBWedxWWXXca+qwr3YdWfOsGyY71c4P187QMrxvWtDD+7m/WHXmPF6z3MG55YszxUt5Al317Lppz5Kzu9gUc6h3nFFmBtrfzpdZdPar212SKtuTNKGJKIYkNyS81Qzt/evqZp3FWmWfaGTIWGv9YY1M2pGXdHv3xLGjJ89P0rg5qB4j7HnR8/P2jiHJS38JrsOUOWZy/2HnHNcfnnHf13y7/yHZhbR9f8VXzo+gsh7/0+O9rf5c7iE/3ZhSajocXL+ns5+9Rhrl9h2SSzezfs38/6kRHW557kv9WMLTo5OqR4U3Rvey5pg9Zl0FqfbT6LmW0/E6XVV6OEIYkoZ2FZbDJiqXbc6bbzTvdzlGMiZaHPEDriZ6pt+qUGMcQdP5nvKzcZH65fyOH6hWOz7cfm++QeNzwMBw7E3ynyrbdgyxboLTDEO3e2ff68ldz5Kw2Tu6lXWtLqq9GwWpEqNt15BWkfH3L+uFFSUz7/iROnk0rMMGM6OuD4xGHGNDbG39t+9Pny5WP3tk9LOf9dNA9DZJYox7yC6dayZuXMaXc4cqT4ve07O2HfvsL3tl++PL7GMvp86dJEm8HK9e+ihCEyS8RNXjMo6/02JMbICHR3l04s3d0Tj507t+CikxMSy4IFlf9cOTRxT2SWOJPmFaSpZNPY8uVwySXxJzh5Evbvj08su3fDk0/C0aMTj124sHATWO5rK1dmE1DKlDBk1poNTSlnyryCNJWaQAgBcxvmzYM1a7KPYo4eLdyfMvrnli3Z54MFJrC2tMQnlg9+MNuxnzAlDJmVZstNgc6UeQVpipurAgnMbVi4EM49N/uIMzICPT3xiaWzE7Ztg66usdn29PUpYYhM1Wy6KVA573EiE5Wau1DxdahqarK1iZYWuOii+P0GB7PNYJ2d2eVWKkAJQ6asmpt8tAidhCq1yGHV9hdlMrBqVfZRITWldxGZaLTJp6NvAOd0k8/mHR1phwbE/8ir9scvqbn16nOpz9QW3Kb+ovGUMGRKijX5VINChYB+/FLIpovbuOf6C2mLLiZqo7kTbY31uvFSHjVJyZRUe5NPpTqLq7lZTsKpnyiMEkYBKgRKmwnzA5IuBGbLSCyRUGqSylPtbfPVQk0+1d8sJ1JuShh5VAiEyW33Nc7M9t5qb5YTKTc1SeVRIRDuTG/3nQnNciLllFgNw8weMrMuM3spZvtGMztsZjujx+2hxyZJwzEllJrl5EyTZJPUw8A1JfbZ4u7ro8ddkzw2ESoEJJSa5eRMk1iTlLs/Y2ZrK33sdGntHpmMM71ZTs4safdhXGFmu4BO4HPuvmeyJzCzm4GbAVavXl2WoFQIiIhMlOYoqe3AGne/CPgqsHkqJ3H3B9y93d3bW1payhqgiIicllrCcPcj7t4fPX8CyJhZc1rxiIhIcaklDDNbYZZdtMXMNkSx9KQVj4iIFJdYH4aZPQJsBJrNbC9wB5ABcPf7gRuAW8xsCBgAbvToBuOFjnX3B5OKVURESktylNRNJbbfB9w3lWNFRKTytDSIiIgEUcIQEZEgShgiIhJECUNERIIoYYiISBAlDBERCaKEISIiQZQwREQkiBKGiIgEUcIQEZEgShgiIhJECUNERIIoYYiISBAlDBERCaKEISIiQZQwREQkiBKGiIgEUcIQEZEgShgiIhJECUNERIIoYYiISBAlDBERCaKEISIiQZQwREQkiBKGiIgEUcIQEZEgShgiIhJECUNERIIoYYiISBAlDBERCZJYwjCzh8ysy8xeitm+0cwOm9nO6HF7zrZrzOw1M3vDzG5LKkYREQmXZA3jYeCaEvtscff10eMuADOrBb4GfBh4H3CTmb0vwThFRCRAYgnD3Z8Beqdw6AbgDXd/y91PAd8BritrcCIiMmlp92FcYWa7zOynZnZ+9Fob8LucffZGr4mISIrmpPje24E17t5vZh8BNgPnAFZgX487iZndDNwMsHr16iTiFBERUqxhuPsRd++Pnj8BZMysmWyNYlXOrmcBnUXO84C7t7t7e0tLS6Ixi4icyVJLGGa2wswser4hiqUHeAE4x8zWmdlc4EbgsbTiFBGRrMSapMzsEWAj0Gxme4E7gAyAu98P3ADcYmZDwABwo7s7MGRmfw48CdQCD7n7nqTiFBGRMJYto2eH9vZ237p1a9phiIjMGGa2zd3bQ/ZNe5SUiIjMEEoYIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJIgShoiIBFHCEBGRIEoYIiISRAlDRESCKGGIiEgQJQwREQmihCEiIkGUMEREJMisuh+GmXUD76QdR6QZOJh2ENMwk+NX7OlQ7OmYbuxr3D3o/tazKmFUEzPbGnpTkmo0k+NX7OlQ7OmoZOxqkhIRkSBKGCIiEkQJIzkPpB3ANM3k+BV7OhR7OioWu/owREQkiGoYIiISRAljmszsGjN7zczeMLPbCmz/lJl1m9nO6PEf0oizEDN7yMy6zOylmO1mZv8QfbYXzeySSscYJyD2jWZ2OOd7v73SMcYxs1Vm9isze8XM9pjZZwrsU5XffWDsVfndm1mdmf3GzHZFsX+pwD7zzOy70ff+vJmtrXykEwXGnnxZ4+56TPEB1AJvAu8B5gK7gPfl7fMp4L60Y42J/18ClwAvxWz/CPBTwIDLgefTjnkSsW8EfpJ2nDGxrQQuiZ4vBF4v8P+mKr/7wNir8ruPvssF0fMM8Dxwed4+fwbcHz2/Efhu2nFPIvbEyxrVMKZnA/CGu7/l7qeA7wDXpRxTMHd/Bugtsst1wDc96zmg0cxWVia64gJir1ruvs/dt0fPjwKvAG15u1Xldx8Ye1WKvsv+6K+Z6JHfiXsd8I3o+Q+AD5qZVSjEWIGxJ04JY3ragN/l/H0vhX88n4iaFX5gZqsqE1pZhH6+anVFVIX/qZmdn3YwhURNHheTvWLMVfXonTWnAAACxElEQVTffZHYoUq/ezOrNbOdQBfwC3eP/d7dfQg4DCytbJSFBcQOCZc1ShjTU+jKIz/r/xhY6+7vB37J6auXmSDk81Wr7WSXPLgI+CqwOeV4JjCzBcCjwGfd/Uj+5gKHVM13XyL2qv3u3X3Y3dcDZwEbzOyCvF2q9nsPiD3xskYJY3r2ArlZ/CygM3cHd+9x95PRX78OXFqh2Mqh5OerVu5+ZLQK7+5PABkza045rDFmliFb4H7L3X9YYJeq/e5LxV7t3z2Au/cBTwPX5G0a+97NbA6wmCpr+oyLvRJljRLG9LwAnGNm68xsLtlOssdyd8hrd/442TbfmeIx4N9FI3YuBw67+760gwphZitG257NbAPZ/+s96UaVFcX1IPCKu/9tzG5V+d2HxF6t372ZtZhZY/S8HvhD4NW83R4D/n30/AbgKY96lNMUEnslypo55T7hmcTdh8zsz4EnyY6Yesjd95jZXcBWd38M+Asz+zgwRPZK5VOpBZzHzB4hO6Kl2cz2AneQ7UzD3e8HniA7WucN4Djwp+lEOlFA7DcAt5jZEDAA3FgNP/zIlcAngd1RmzTAF4DVUPXffUjs1frdrwS+YWa1ZJPY99z9J3m/1weB/21mb5D9vd6YXrjjhMSeeFmjmd4iIhJETVIiIhJECUNERIIoYYiISBAlDBERCaKEISIiQZQwRCrIzN6utklsIqGUMEREJIgShkhCzGyzmW2L7l9wc9rxiEyXZnqLJOfT7t4bLeXwgpk9mnZAItOhhCGSnL8wsz+Knq8CzkkzGJHpUsIQSYCZbSS7QNwV7n7czJ4G6lINSmSa1IchkozFwKEoWZxH9jarIjOaEoZIMn4GzDGzF4G/AZ5LOR6RadNqtSIiEkQ1DBERCaKEISIiQZQwREQkiBKGiIgEUcIQEZEgShgiIhJECUNERIIoYYiISJD/D/2Pev0+XAsoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c394336d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# put the plots together\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.plot(glass.al, glass.ri_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Refresher: interpreting linear regression coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression equation: $y = \\beta_0 + \\beta_1x$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51699012])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the equation\n",
    "linreg.intercept_ + linreg.coef_ * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51699012])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the predict method\n",
    "linreg.predict(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['al'] [-0.00247761]\n"
     ]
    }
   ],
   "source": [
    "# examine coefficient for al\n",
    "print(feature_cols, linreg.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 0.0025 unit decrease in 'ri'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5145125136125304"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) decreases ri by 0.0025\n",
    "1.51699012 - 0.0024776063874696243"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51451251])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=3 using the predict method\n",
    "linreg.predict(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting a Categorical Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    70\n",
       "2    76\n",
       "3    17\n",
       "5    13\n",
       "6     9\n",
       "7    29\n",
       "Name: glass_type, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine glass_type\n",
    "glass.glass_type.value_counts().sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  \n",
       "id                        \n",
       "22   1.521227          0  \n",
       "185  1.521103          1  \n",
       "40   1.520781          0  \n",
       "39   1.520781          0  \n",
       "51   1.520682          0  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# types 1, 2, 3 are window glass\n",
    "# types 5, 6, 7 are household glass\n",
    "glass['household'] = glass.glass_type.map({1:0, 2:0, 3:0, 5:1, 6:1, 7:1})\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change our task, so that we're predicting **household** using **al**. Let's visualize the relationship to figure out how to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAF4JJREFUeJzt3X2UXPV93/H3V6NRWGEb4SCfgB4sTBRSbMDCG8AlpyW1XTBNQHZ4UkptbDc6TUrdNi2nOKXgELv2iY6d2A2JQx38mAo/4CgKR65O05jTNC1EizFgIMpRAVsPtMgEYcdSzGr17R8z+/Nodnbn7sPVjOD9OmfPzr33N7/53t/emc/OfZiJzESSJIBFgy5AkjQ8DAVJUmEoSJIKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJhaEgSSoWD7qA2TrllFNyzZo1gy5Dko4rDzzwwHcyc3m/dsddKKxZs4axsbFBlyFJx5WI+FaVdu4+kiQVhoIkqTAUJEmFoSBJKgwFSVJhKEiSCkNBklTUFgoRcWdEPBMR35xmeUTExyNiV0Q8HBHn1VWLJKmaOi9e+zTw28Bnp1n+VmBt++cC4Hfbv4+ZLQ/uZdP2new7cIjTlo1w4yVnsn7diqHrc1jdvOURNt+/m4lMGhFc+JqTeerZQz3Xvcq4VB27znYnjTSJgOcOjk9pt2LZCD/zk8v52l/uZ++BQwQw+Y3kJy9tcuvPvfao/if73XvgEI0IJjJZNtLk+z8YZ/zI0X0vbbb+nzrYXtDdX3eN3X1M1jKbGnuND9BzLCbrX9Exjt33n3zc6aa7x7/XuB84OF5pO9/y4F7ev/VRDhwan7Ju/darXy2zeZ7NVMewOtavKZGZ/VvNtfOINcA9mfm6Hst+D7g3Mze3p3cCF2fm0zP1OTo6mgtxRfOWB/fyvq88wqHxiTJvpNngQ28/e84DXkefw+rmLY/w+fu+PWObyXUH+o5L1bHr1W6umo1g05Xnlhem+fY72R9MXd86+mwuCggYn5j5OTzSbPDzb1jB3Q/snVVNnePfb3xm2s63PLiXG7/0EONHjq6z2Qiu+alVU+rqtV79aqnyPJupjsntYNgs5GtKRDyQmaP92g3ymMIKYHfH9J72vGNi0/adUzbwQ+MTbNq+c6j6HFab79/dt83kulcZl6pj16vdXI1PZOl/Ifqd7K+OGnv1OX4k+wYCtMZx8/27Z11T5/j3W6eZtvNN23dOeSGG1rr1qqvXevWrpcrzbKY6hvU5OojXlEF+9lH0mNdzC4+IjcBGgNWrVy/Ig+87cGhW8wfV57CaqPgOc6Z171xWdewWeiwn+1uofuv4Wy9En1X/XtM9dpUa5rL9z6aufrX0q7HqtjhMBvGaMsh3CnuAVR3TK4F9vRpm5h2ZOZqZo8uX9/2Qv0pOWzYyq/mD6nNYNaJXpk912rKRSuNSdewWeiwn+1uofmda30H2WfXv1euxO39XaVt1/mzr6ldLvxpnWj6sz9FBvKYMMhS2Au9on4V0IfB8v+MJC+nGS85kpNk4at5Is1EOcg1Ln8NqwwWr+raZXPcq41J17Hq1m6tmI0r/C9HvZH911Nirz+aioNno/6I60myw4YJVs66pc/z7rdNM2/mNl5zZOk7QpdmInnX1Wq9+tVR5ns1Ux7A+RwfxmlLb7qOI2AxcDJwSEXuAW4EmQGZ+AtgGXAbsAg4C76qrll4mD9Is5FH9OvocVh9Y3zqAXPXsI5h5XKqOXXe7hTr7qLPfhTj7qLvGhTj7qNf4TDcWvc4+Gn31K+d89tF0417l7KPJ+dOd9dNdV7+zj+b6POtXxzAaxGtKrWcf1WGhzj6SpJeS4+HsI0nSkDEUJEmFoSBJKgwFSVJhKEiSCkNBklQYCpKkwlCQJBWGgiSpMBQkSYWhIEkqDAVJUmEoSJIKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJhaEgSSoMBUlSYShIkgpDQZJUGAqSpMJQkCQVhoIkqTAUJEmFoSBJKgwFSVJhKEiSCkNBklTUGgoRcWlE7IyIXRFxU4/lqyPiaxHxYEQ8HBGX1VmPJGlmtYVCRDSA24G3AmcBGyLirK5mNwNfzMx1wLXA79RVjySpvzrfKZwP7MrMJzLzBeAu4IquNgm8on37JGBfjfVIkvqoMxRWALs7pve053V6P3BdROwBtgH/oldHEbExIsYiYmz//v111CpJot5QiB7zsmt6A/DpzFwJXAZ8LiKm1JSZd2TmaGaOLl++vIZSJUlQbyjsAVZ1TK9k6u6h9wBfBMjM/w2cAJxSY02SpBnUGQo7gLURcXpELKF1IHlrV5tvA28CiIi/QysU3D8kSQNSWyhk5mHgBmA78Dits4wejYjbIuLydrN/A/xiRDwEbAauz8zuXUySpGNkcZ2dZ+Y2WgeQO+fd0nH7MeCiOmuQJFXnFc2SpMJQkCQVhoIkqTAUJEmFoSBJKgwFSVJhKEiSCkNBklQYCpKkwlCQJBWGgiSpMBQkSYWhIEkqDAVJUmEoSJIKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJhaEgSSoMBUlSYShIkgpDQZJUGAqSpMJQkCQVhoIkqTAUJEmFoSBJKhbPtDAiHgFyuuWZeU6f+18KfAxoAJ/MzA/3aHM18P724zyUmb/Qv2xJUh1mDAXgZ9u//3n79+fav/8xcHCmO0ZEA7gdeAuwB9gREVsz87GONmuB9wEXZeZzEfGqWdYvSVpAM4ZCZn4LICIuysyLOhbdFBF/Dtw2w93PB3Zl5hPtPu4CrgAe62jzi8Dtmflc+/Gemf0qSJIWStVjCidGxE9PTkTE3wVO7HOfFcDujuk97XmdfgL4iYj484i4r727SZI0IP12H016D3BnRJzUnj4AvLvPfaLHvO7jE4uBtcDFwErgzyLidZl54KiOIjYCGwFWr15dsWRJ0mxVCoXMfAA4NyJeAURmPl/hbnuAVR3TK4F9Pdrcl5njwJMRsZNWSOzoevw7gDsARkdHpz3wLUman35nH/3KNPMByMyPznD3HcDaiDgd2AtcC3SfWbQF2AB8OiJOobU76YlKlUuSFly/dwovn2vHmXk4Im4AttM6JfXOzHw0Im4DxjJza3vZP4yIx4AJ4MbMfHaujylJmp/IPL72xoyOjubY2Nigy5Ck40pEPJCZo/3aVTr7KCJWRsQfRsQzEfH/IuLuiFg5/zIlScOk6impnwK2AqfROq30j9vzJEkvIlVDYXlmfiozD7d/Pg0sr7EuSdIAVA2F70TEdRHRaP9cB3hAWJJeZKqGwruBq4H/CzwNXEn/i9ckSceZqhevfRu4vOZaJEkDVikUImI5rQ+vW9N5n8z03YIkvYhU/eyjPwL+DPgTWheZSZJehKqGwtLM/He1ViJJGriqB5rviYjLaq1EkjRw/T4Q73u0Pu46gF+NiBeAF9rTmZmvqL9ESdKx0u+b1+b8gXiSpONP1c8+ivbFa/+hPb0qIs6vtzRJ0rFW9ZjC7wBv5Iffh/A3wO21VCRJGpiqZx9dkJnnRcSDAJn5XEQsqbEuSdIAVH2nMB4RDdrfsdy+mO1IbVVJkgaiaih8HPhD4FUR8UHgfwL/sbaqJEkDUfWzj/4gIh4A3kTrdNT1mfl4rZVJko65qmcfnQE8mZm3A98E3hIRy2qtTJJ0zFXdfXQ3MBERPw58Ejgd+C+1VSVJGoiqoXAkMw8Dbwc+lpn/Gji1vrIkSYMwm7OPNgDvAO5pz2vWU5IkaVCqhsK7aF289sHMfDIiTgc+X19ZkqRBqHr20WPAezumnwQ+XFdRkqTBqPrNa0/SvnCtU2a+ZsErkiQNTNWPuRjtuH0CcBXwyoUvR5I0SJWOKWTmsx0/ezPzt4B/UHNtkqRjrOruo/M6JhfReufgdy1I0otM1d1HH+m4fRh4Crh6wauRJA1U1bOPfqbuQiRJg1f1s49OioiPRsRY++cjEXFShftdGhE7I2JXRNw0Q7srIyIjYnS6NpKk+lW9eO1O4Hu0dhldDXwX+NRMd2h//8LtwFuBs4ANEXFWj3Yvp3UNxP3Vy5Yk1aFqKJyRmbdm5hPtn18D+l2jcD6wq93+BeAu4Ioe7X4d+A3gbytXLUmqRdVQOBQRPz05EREXAYf63GcFsLtjek97XhER64BVmXkPM4iIjZO7rvbv31+xZEnSbFU9++iXgM90HEd4Dnhnn/tEj3nlquiIWAT8JnB9vwfPzDuAOwBGR0enXFktSVoYVUPhcVq7eM4AlgHPA+uBh2e4zx5gVcf0SmBfx/TLgdcB90YEwI8BWyPi8swcq1iXJGkBVQ2FPwIOAF8H9la8zw5gbfsTVfcC1wK/MLkwM58HTpmcjoh7gX9rIEjS4FQNhZWZeelsOs7MwxFxA7AdaAB3ZuajEXEbMJaZW2dZqySpZlVD4X9FxNmZ+chsOs/MbcC2rnm3TNP24tn0LUlaeDOGQkQ8Quvg8GLgXRHxBPADWgeRMzPPqb9ESdKx0u+dws8ekyokSUNhxlDIzG8dq0IkSYNX9eI1SdJLgKEgSSoMBUlSYShIkgpDQZJUGAqSpMJQkCQVhoIkqTAUJEmFoSBJKgwFSVJhKEiSCkNBklQYCpKkwlCQJBWGgiSpMBQkSYWhIEkqDAVJUmEoSJIKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJhaEgSSpqDYWIuDQidkbEroi4qcfyX4mIxyLi4Yj47xHx6jrrkSTNrLZQiIgGcDvwVuAsYENEnNXV7EFgNDPPAb4M/EZd9UiS+qvzncL5wK7MfCIzXwDuAq7obJCZX8vMg+3J+4CVNdYjSeqjzlBYAezumN7Tnjed9wBfrbEeSVIfi2vsO3rMy54NI64DRoG/P83yjcBGgNWrVy9UfZKkLnW+U9gDrOqYXgns624UEW8G/j1weWb+oFdHmXlHZo5m5ujy5ctrKVaSVG8o7ADWRsTpEbEEuBbY2tkgItYBv0crEJ6psRZJUgW1hUJmHgZuALYDjwNfzMxHI+K2iLi83WwT8DLgSxHxjYjYOk13kqRjoM5jCmTmNmBb17xbOm6/uc7HlyTNjlc0S5IKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJhaEgSSoMBUlSYShIkgpDQZJUGAqSpMJQkCQVhoIkqTAUJEmFoSBJKgwFSVJhKEiSCkNBklQYCpKkwlCQJBWGgiSpMBQkSYWhIEkqDAVJUmEoSJIKQ0GSVBgKkqTCUJAkFYaCJKkwFCRJxeI6O4+IS4GPAQ3gk5n54a7lPwJ8FngD8CxwTWY+VUctWx7cy6btO9l34BCnLRvhxkvOZP26FbPq4+Ytj7D5/t1MZNKI4MLXnMxTzx46qk+A9299lAOHxgE4eWmTW3/uteWxtjy496jlk23+0Tmncs9DT5f5zUUwfmRqDUubixifOFKWLQo4Y/mJ7Hrm+2SPmk9c0uCDbzv7qHXtXo8NF6xi9NWvnFLX8W5JI3hhYuqonLy0yXMHx2lEMJE/XB7A0iUNvv/CRFm2bKTJC4cnONge8M6/Z/ffclHAkYQVc9y+JrfRvQcOlcef/D3XPqXZisxeLyUL0HFEA/gr4C3AHmAHsCEzH+to88vAOZn5zyLiWuBtmXnNTP2Ojo7m2NjYrGrZ8uBe3veVRzg0PlHmjTQbfOjtZ1d+kt285RE+f9+3Z2zTXNR6Ah/pGtJmI9h05bkA3PilhxjvblCzxqLgI1edy/p1K6Zdj4CeoaKpmo3gmp9axRf+Yve0f8vZbl+9ttH59il1iogHMnO0X7s6dx+dD+zKzCcy8wXgLuCKrjZXAJ9p3/4y8KaIiIUuZNP2nVOebIfGJ9i0fWflPjbfv7tvm/EjUwMBYHwi2bR9J5u27zzmgQAwcSTLuk63HgZCdeMTyeb7pw8EmP321WsbnW+f0lzUGQorgM5XoD3teT3bZOZh4HngR7s7ioiNETEWEWP79++fdSH7Dhya1fxeJub5jmrfgUOzeryFNvnY810PtVQZx9n8vau2HeQ2pJeGOkOh13/83c+kKm3IzDsyczQzR5cvXz7rQk5bNjKr+b005vkG5rRlI7N6vIU2+djzXQ+1VBnH2fy9q7Yd5Dakl4Y6Q2EPsKpjeiWwb7o2EbEYOAn464Uu5MZLzmSk2Thq3kizUQ4MV7HhglV92zQXBYt6vFY0G8GNl5zJjZecSbNXg5o1FkVZ1+nWw6iortloHZyf6W852+2r1zY63z6luagzFHYAayPi9IhYAlwLbO1qsxV4Z/v2lcCfZg1HvtevW8GH3n42K5aNELTODpntAbsPrD+b6y5cXf5DbERw0RmvPKrPTVedy0evfj3LRprlficvbbLpytZB3vXrVrDpqnOPWj7Z5roLVx81vznNX2Zpc9FRyxYFrH3VidO+qJ+4pFEOMk+3HtdduJrfvOb1U+o63i1p9B6Vk5e21rP7v/2gNV6dy5aNNFnaMeCTf88PrD97yt9yMiPmsn11bqOdjz/5ey59SnNR29lHABFxGfBbtE5JvTMzPxgRtwFjmbk1Ik4APgeso/UO4drMfGKmPudy9pEkvdRVPfuo1usUMnMbsK1r3i0dt/8WuKrOGiRJ1XlFsySpMBQkSYWhIEkqDAVJUmEoSJIKQ0GSVBgKkqSi1ovX6hAR+4FvDbqODqcA3xl0EXNk7YNxPNcOx3f9L+XaX52ZfT887rgLhWETEWNVrhIcRtY+GMdz7XB812/t/bn7SJJUGAqSpMJQmL87Bl3APFj7YBzPtcPxXb+19+ExBUlS4TsFSVJhKFQUEZdGxM6I2BURN/VYfn1E7I+Ib7R//ukg6uwWEXdGxDMR8c1plkdEfLy9Xg9HxHnHusbpVKj94oh4vmPMb+nVbhAiYlVEfC0iHo+IRyPiX/ZoM5RjX7H2YR77EyLiLyLioXb9v9ajzY9ExBfaY39/RKw59pVOVbH2el9rMtOfPj+0viTo/wCvAZYADwFndbW5HvjtQdfao/a/B5wHfHOa5ZcBX6X1xWMXAvcPuuZZ1H4xcM+g65ymtlOB89q3Xw78VY9tZijHvmLtwzz2AbysfbsJ3A9c2NXml4FPtG9fC3xh0HXPovZaX2t8p1DN+cCuzHwiM18A7gKuGHBNlWTm/2Dm772+AvhsttwHLIuIU49NdTOrUPvQysynM/Pr7dvfAx4Hur9LcyjHvmLtQ6s9nn/Tnmy2f7oPnl4BfKZ9+8vAmyJi4F9VXrH2WhkK1awAdndM76H3k+Tn27sBvhwRq45NafNWdd2G1Rvbb7W/GhGvHXQxvbR3Tayj9V9fp6Ef+xlqhyEe+4hoRMQ3gGeA/5aZ0459Zh4Gngd+9NhW2VuF2qHG1xpDoZpe/0F0p/cfA2sy8xzgT/jhfyHDrsq6Dauv07p0/1zgPwFbBlzPFBHxMuBu4F9l5ne7F/e4y9CMfZ/ah3rsM3MiM18PrATOj4jXdTUZ2rGvUHutrzWGQjV7gM40Xgns62yQmc9m5g/ak/8ZeMMxqm2++q7bsMrM706+1c7W94E3I+KUAZdVREST1ovqH2TmV3o0Gdqx71f7sI/9pMw8ANwLXNq1qIx9RCwGTmLIdlVOV3vdrzWGQjU7gLURcXpELKF1YGprZ4OufcGX09oPezzYCryjfSbMhcDzmfn0oIuqIiJ+bHI/cEScT2t7fnawVbW06/p94PHM/Og0zYZy7KvUPuRjvzwilrVvjwBvBv6yq9lW4J3t21cCf5rto7iDVKX2ul9rFi9kZy9WmXk4Im4AttM6E+nOzHw0Im4DxjJzK/DeiLgcOEzrP47rB1Zwh4jYTOtMkVMiYg9wK62DV2TmJ4BttM6C2QUcBN41mEqnqlD7lcAvRcRh4BBw7TA8sdsuAv4J8Eh7/zDArwKrYejHvkrtwzz2pwKfiYgGrbD6Ymbe0/V8/X3gcxGxi9bz9drBlXuUKrXX+lrjFc2SpMLdR5KkwlCQJBWGgiSpMBQkSYWhIEkqDAWpBhHx1DBezCX1YyhIkgpDQZqniNgSEQ+0P/9+46DrkebDK5ql+Xt3Zv51+2MJdkTE3YMuSJorQ0Gav/dGxNvat1cBawdZjDQfhoI0DxFxMa0PLXtjZh6MiHuBEwZalDQPHlOQ5uck4Ll2IPwkra/VlI5bhoI0P/8VWBwRDwO/Dtw34HqkefFTUiVJhe8UJEmFoSBJKgwFSVJhKEiSCkNBklQYCpKkwlCQJBWGgiSp+P8mX2tnjjcI8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c30a1b9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(glass.al, glass.household)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's draw a **regression line**, like we did before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fit a linear regression model and store the predictions\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "linreg.fit(X, y)\n",
    "glass['household_pred'] = linreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VPW9//HXhzRUlCIqWGUrWvlpQcAlD3CrYt3QXsHrVum11RaLS12q1iu2VCtutCgWLBVRqUAv0AoWggtbRUGpkbAbbHpRqiwqqYiUCyokn98f34GGMElOkpk5M8n7+XjkkTkz3znnM2cm88n3fDdzd0RERKJoFncAIiKSO5Q0REQkMiUNERGJTElDREQiU9IQEZHIlDRERCQyJQ0REYlMSUNERCJT0hARkci+FHcAqdamTRvv3Llz3GGIiOSUJUuW/NPd29ZWrtEljc6dO1NcXBx3GCIiOcXM3otSTpenREQkMiUNERGJTElDREQiU9IQEZHIlDRERCQyJQ0REYlMSUNERCJT0hARyXVFRfDggxk5lJKGiEiu2r4dbr8dTjkFnngCPv007YdU0hARyUWvvAI9esCIEXDttbBqFRx4YNoPq6QhIpJLtm6F666DM88Es5A8fvc7aNUqI4dX0hARyRUvvgjdusGTT4bLUitWwBlnZDQEJQ0RkWz38cfw/e/Dt78dLkH99a/w8MOw//4ZDyXWpGFm48xsk5m9Vc3j/2VmKxM/i8ysZ6ZjFBGJ1dSp0LUrTJ4Md98NS5ZAr16xhRN3TeMZoG8Nj68FznD3HsB9wNhMBCUiErsPP4RLLoHLLoOOHaG4GO69F7785VjDijVpuPsCYHMNjy9y908Sm28AHTISmIhIXNxhwoRQu3jhBRg2DN54A3pmx4WWXFqEaSDwUtxBiIikzfvvh+6zs2bBqafC00/D0UfHHdVe4r48FYmZnUlIGndW8/ggMys2s+KysrLMBici0lAVFfD446Fn1MKF8NhjsGBB1iUMyIGkYWY9gKeA/u7+cbIy7j7W3QvcvaBt21qXuBURyR5r1sC3vgU33AAnnQRvvQU33gjNsvPrOTujSjCzTsBzwPfc/e9xxyMikjLl5fDII2FU9/Ll4VLUnDnQuXPckdUo1jYNM5sM9AHamNl64B4gH8DdxwB3A4cAvzMzgF3uXhBPtCIiKVJSAj/8Ibz5JvTvH0Z0t2sXd1SRxJo03H1ALY9fA1yToXBERNLriy9Cb6j77w+D9KZMgcsvD9OB5Ihc6j0lIpK7liwJtYuVK2HAABg5EnKwDTar2zRERHLejh0weDD07g3//CcUFsKkSTmZMEA1DRGR9HntNRg4EP7+d7jmGhg+HFq3jjuqBlFNQ0Qk1bZtg5tugtNPD+0Yc+eGmWlzPGGAkoaISGrNnQvdu8Po0SFxrFoFZ58dd1Qpo6QhIpIKW7aES1HnnhsmFVy4MDR2t2wZd2QppaQhItJQM2aECQbHj4e77gqD9U49Ne6o0kIN4SIi9VVWBjffHMZb9OgBzz8PJ5wQd1RppZqGiEhduYdFkbp2hWnTYOhQWLy40ScMUE1DRKRuNmyA66+HmTPDCnrjxoXZaZsI1TRERKJwh6eeCgli3rww2eCiRU0qYYBqGiIitVu7FgYNCsnijDNC8jjqqLijioVqGiIi1amogFGj4NhjoagoLJT08stNNmGAahoiIsn97W9h3MWiRXD++fDEE9CxY9xRxU41DRGRynbtCtOXH3ccvP02TJgAL7yghJGgmoaIyG4rVoTpy5cuhUsugd/+Fg47LO6oskrcK/eNA/4D2OTuxyZ53ICRwAXAduBqd1+aqfimL9vA8NmlbNyyg3atW3DHeUdz0fHts26f2WrI9FVMLlpHuTt5Zpx05EH84+Md1b722s5NXc7d7rIbtuwgz4xy933KtG/dgjOPacv8v5WxccsODmyRzxe7ytm+swKAg/bP554Lu+05RrJ9tq7ynMrym0G5Q4VDnhkDenfk/ou6J30tZx7TlmlL1rOj0n4McKB1i3zM4JPtO/fclyy+6vZb+fXt3s/u+NtXOY9Rnr9l+85qz3/l50cpX9P7VTm2muKKEkutf2uffw7330/FsGF8st9X+PlFdzG3y2mU/2bJPuco22T6O8U8yR9TppjZ6cA2YEI1SeMC4CZC0ugNjHT33jXts6CgwIuLixsc2/RlG7jruVXs2Fm+574W+Xk8dHH3er8h6dhnthoyfRV/eOP9GstUfu21nZu6nLtkZesrP88YfmlPgJTs88qTOlHwtYNTHl915yiK3ecR6vYaq57/2o5fn/erRX4el5zYnmlLNtQYV5RYqv1bKyoKtYvVq5ne/SzuOfMaPm3xlUixxy2V3ylmtiTKctqxtmm4+wJgcw1F+hMSirv7G0BrMzs8E7ENn126z4d0x85yhs8uzap9ZqvJRetqLVP5tdd2bupy7pKVra+d5c7w2aUp2+fkonVpiQ/q/7p3n8e6Pr/q+a/t+fV5v3bsLGdy0bpa44oSyz7H374dbr8dTjkFtm7l9qsf5CcX3LpPwqgp9rjF8Z2S7Q3h7YHK3z7rE/ftxcwGmVmxmRWXlZWl5MAbt+yo0/1x7TNbJbsclMzu117buanLuUv1+dy4ZUfK9lnunpb4Kv+u7z7q8/zKz4ny/Pq8X3X9LNW0zz33v/JKmCtqxIgw/qKkhOe+2iPy/rNFHN8p2Z40kq22vs8nyN3HunuBuxe0TdESiu1at6jT/XHtM1vlWbK3bl+7X3tt56Yu5y7V57Nd6xYp22eeWVriq/y7vvuoz/MrPyfK8+vzftX1s1TTPo/arwKuuw7OPDPcMX9+GHvRqlWtcWTj32kc3ynZnjTWA5X7uXUANmbiwHecdzQt8vP2uq9Ffh53nHd0Vu0zWw3oXXv3xMqvvbZzU5dzl6xsfeXnGXecd3TK9jmgd8e0xAf1f927z2Ndn1/1/Nf2/Pq8Xy3y8xjQu2OtcUWJ5bz3ljBjzLVhBb3bb4eVK6FPn8hxZOPfaRzfKdne5bYQuNHMphAawj919w8yceDdjUip7JWQjn1mq929hKL2nqrt3NTl3FUum8reU8n2Wd/eU1VfSyp6TyU7R3XtPRXl+dX1hqp6/Ki9p6p7vyrHVvC1g+vUe6ryPrd/8BEPLvw95y+bG2alnfFn6L1vf5oocWSbOL5T4u49NRnoA7QBPgLuAfIB3H1Mosvtb4G+hC63P3D3GrtGpar3lIg0AlOnwo9/DJs3h8WRfv7zsKqe7CNq76lYaxruPqCWxx34cYbCEZHG4sMPQ7J47rmwxsWcOdCzZ9xRNQrZ3qYhIhKde5j2o2vXMPXHsGFhHIYSRspke5uGiEg0778P114Ls2aF9bmffhqOzr7G61ynmoaI5LaKitBttls3WLgwTGW+YIESRpqopiEiuWvNGrjmGnj1VTj77NCdtnPnuKNq1FTTEJHcU14ellvt0QOWLw8r6c2Zo4SRAappiEhuKSkJEwy++Sb06xcuTbVrF3dUTYZqGiKSG774AoYOheOPh3ffhcmTYfp0JYwMU01DRLLfkiWhdrFyJQwYACNHQormmZO6UU1DRLLXjh0weHCY9qOsDGbMgEmTlDBipJqGiGSn116DgQPh738Pvx9+GFq3jjuqJk81DRHJLtu2wU03wemnh3aMuXND7ygljKygpCEi2WPuXOjeHUaPhhtvhFWrwvgLyRpKGiISvy1bwiWoc88Ns9DuHtndsmXckUkVShoiEq8ZM8IEg+PHh0bv5cvD3FGSldQQLiLxKCuDm2+GKVPCyO6ZM+HEE+OOSmoRa03DzPqaWamZrTGzwUke72Rm881smZmtNLML4ohTRFLIPQzM69oVpk0LA/YWL1bCyBGxJQ0zywNGA+cDXYEBZta1SrEhwJ/c/XjgCuB3mY1SRFJqwwbo3x+++1048khYtgx+8Qto3jzuyCSiOGsavYA17v6uu38BTAH6VynjQKvE7QOBjRmMT0RSxT2sb9GtG8ybFyYbXLQobEtOibNNoz2wrtL2eqDqau+/BOaY2U3AAYD63onkmrVrYdCgkCzOOCOMuTjqqLijknqKs6ZhSe7zKtsDgGfcvQNwATDRzPaJ2cwGmVmxmRWXlZWlIVQRqbOKCnjssTDuoqgozEb78stKGDkuzqSxHuhYabsD+15+Ggj8CcDd/wrsB7SpuiN3H+vuBe5e0FZz0ojEr7Q0jOi++ebwu6QErrsOmqmXf66L8x1cDHQxsyPMrDmhobuwSpn3gbMAzOwbhKShqoRIttq1C4YNg549YfVqmDABXngBOnas/bmSE2Jr03D3XWZ2IzAbyAPGuXuJmQ0Fit29ELgdeNLMbiVcurra3atewhKRbLBiRZi+fOlSuOQS+O1v4bDD4o5KUizWwX3u/iLwYpX77q50ezWgoaEi2ezzz+GBB+Chh+CQQ2Dq1JA0pFHSiHARqb+iolC7WL0avv99ePRROPjguKOSNFKrlIjU3fbtcPvtcMopsHUrvPhimDtKCaPRU01DROrm1VfDjLTvvBN6RP3qV9CqVe3Pk0ZBNQ0RiWbrVrj+eujTJ2zPnx/GXihhNClKGiJSu5degmOPhbFjw2WplSv/nTykSdHlKRGp3ubN8JOfwMSJYVbaRYugd9XZfqQpUU1DRJKbNi0kismTw0y0S5cqYYhqGiJSxYcfhvW5p02DE06A2bPDCG8RVNMQkd3cw7QfXbvC88+H6UCKipQwZC+qaYgIrFsH114bGrxPPTWsfXH00XFHJVlINQ2RpqyiAsaMCYshLVgAo0aF30oYUg3VNESaqjVr4JprwmC9s8+GJ5+Ezp3jjkqynGoaIk1NeXlYbrVHD1i+PKykN2eOEoZEUmNNw8xWse9qenu4e4+URyQi6VNSEqYAKSqCfv3CiO527eKOSnJIbZen/iPx+8eJ3xMTv/8L2J6WiEQk9XbuDL2h7rsPDjwwjL34znfAkq26LFK9GpOGu78HYGanunvldS0Gm9nrwNB0BiciKbBkSZi+fOVKGDAARo4ELYss9RS1TeMAMztt94aZnQIc0NCDm1lfMys1szVmNriaMpeb2WozKzGzSQ09pkiT8dlncNddYRR3WRnMmAGTJilhSINE7T01EBhnZgcmtrcAP2zIgc0sDxgNnAOsBxabWWFitb7dZboAdwGnuvsnZnZoQ44p0mS8/npouygtDb8ffhhat447KmkEIiUNd18C9DSzVoC5+6cpOHYvYI27vwtgZlOA/sDqSmV+BIx2908ScWxKwXFFGq9t2+BnPwvrc3/tazB3buhOK5IitfWeuq2a+wFw9xENOHZ7YF2l7fVA1dnQ/l/ieK8DecAv3X1WkngGAYMAOnXq1ICQRHLYvHnwox/Be++FuaMefBBatow7KmlkaqtpfCWNx07WbaNq994vAV2APkAHYKGZHevuW/Z6kvtYYCxAQUFBtV2ERRqlLVvgpz/999QfCxeGqUBE0qC23lP3pvHY64GOlbY7ABuTlHnD3XcCa82slJBEFqcxLpHcUVgYVtP76CMYPBjuuQf22y/uqKQRi9R7ysw6mNmfzWyTmX1kZtPMrEMDj70Y6GJmR5hZc+AKoLBKmenAmYkY2hAuV73bwOOK5L6ystB9tn9/aNMmDNZ76CElDEm7qF1uf0/4Qm9HaIuYmbiv3tx9F3AjMBt4G/iTu5eY2VAz65coNhv42MxWA/OBO9z944YcVySnucOUKWH68mnTYOhQWLwYTjwx7sikiTD32psAzGy5ux9X233ZoKCgwIuLi+MOQyT1Nm4Ml6IKC6FXLxg3LsxOK5ICZrbE3QtqKxe1pvFPM7vSzPISP1cC+o9fJBPcQyN3166hC+0jj4S1upUwJAZRk8YPgcuBD4EPgEtp4OA+EYlg7Vo499wwhflxx4WpQG67DfLy4o5Mmqiog/veB/rVWlBEUqOiAkaPDtOAmIXZaAcNgmZazUDiFSlpmFlbwujszpWf4+6qbYik2u6pP15/Hfr2hSeeAA1alSwRde6pGcBCYB5Qnr5wRJqwXbvCHFG//CXsvz+MHw/f+56mL5esEjVp7O/ud6Y1EpGmbMWKMH350qVwySVh7qjDDos7KpF9RL1A+ryZXZDWSESaos8/h7vvhoICWL8enn0Wpk5VwpCsVduEhf8izAdlwM/M7Avgi8S2u3ur9Ico0kgVFYXaxerV4TLUo4/CIYfEHZVIjWqbeyqdExaKNE3bt8MvfgG/+U1Yn/uFF+ACVeQlN0Sde8oSg/t+kdjuaGa90huaSCP06qvQoweMGBG60JaUKGFITonapvE74GTgu4ntbYRV90Qkiq1bwxQgffqE7fnzw9iLVrrCK7klatLo7e4/Bj4DSKyk1zxtUYk0Ji+9BMceC2PHhtHcK1f+O3mI5JioXW53Jtb0dtgz2K8ibVGJNAabN8NPfgITJ4Z5oxYtgt5VF6cUyS1RaxqjgD8Dh5rZA8BrwINpi0ok102bFhLF5MkwZEgYf6GEIY1A1Lmn/sfMlgBnEbrbXuTub6c1MpFc9OGHYX3uadPghBNg9mzo2TPuqERSJmrvqa8Da919NPAWcI6ZtW7owc2sr5mVmtkaMxtcQ7lLzczNrNa53kVi4Q4TJoTaxfPPh1X0ioqUMKTRiXp5ahpQbmZHAU8BRwCTGnLgRBvJaOB8oCswwMy6Jin3FeBmoKghxxNJm3Xr4Nvfhquugm98A5YvD+t1fylqk6FI7oiaNCoSy7NeDIx091uBwxt47F7AGnd/192/AKYA/ZOUuw/4NYmeWyJZo6ICxowJiyG9+iqMHAkLFsAxx8QdmUjaRE0aO81sAPB94PnEffkNPHZ7YF2l7fWJ+/Yws+OBju7+PCLZZM0a+Na3wtiLXr3grbfg5pu1OJI0elGTxg8Ig/secPe1ZnYE8IcGHjvZfM97Fiw3s2bAo8Dtte7IbJCZFZtZcVlZWQPDEqlBeXlYbrVHj3AZ6qmnwhKsRxwRd2QiGRG199RqQrvC7u21wLAGHns90LHSdgdgY6XtrwDHAq9YWE/gMKDQzPq5e3GV+MYCYwEKCgockXQoKQmLIxUVwYUXhhHd7dvX/jyRRiTqyn1rqVQL2M3dj2zAsRcDXRK1lg3AFfx7mhLc/VOgTaUYXgF+WjVhiKTdzp0wbBjcdx8ceCBMmgRXXKHFkaRJitq9o3JX1/2Ay4CDG3Jgd99lZjcCs4E8YJy7l5jZUKDY3Qsbsn+RlFiyJExfvnJlSBSjRkHbtnFHJRIbc6/f1Rwze83dT0txPA1WUFDgxcWqjEgDffYZ3HsvDB8Ohx4aekn16xd3VCJpY2ZL3L3WsXBRL0+dUGmzGaHmobU2pHF6/fXQdlFaGmoZjzwCrRs8llWkUYh6eeqRSrd3Af8ALk95NCJx2rYNfvazsD53p04wZw6cc07cUYlklai9p85MdyAisZo3D370I3jvvTB31IMPQsuWcUclknWizj11oJmN2D0WwsweMbMD0x2cSNpt2QLXXBNqFM2bhxHdo0YpYYhUI+rgvnHAvwiXpC4HtgK/T1dQIhlRWBimAHnmGbjzzjBY77Ss69shklWitml83d0vqbR9r5ktT0dAImlXVham/JgyJYzsLiyEE0+MOyqRnBC1prHDzPb8C2ZmpwI70hOSSJq4h0TRtWtY72LoUFi8WAlDpA6i1jSuB8ZXasf4BLgqPSGJpMHGjWFywcLCMMHg00+HdbtFpE6iJo23CdOTfx1oDXwKXASsTFNcIqnhDuPGwe23w+efw8MPh3W7NRutSL1ETRozgC3AUsI8USLZb+1aGDQodKc944wwI+1RR8UdlUhOi5o0Orh737RGIpIqFRUwejTcdVeYVPDxx0PyaBa1CU9EqhP1r2iRmXVPayQiqVBaCqefHnpHffObYTrz665TwhBJkRprGma2ijAl+peAH5jZu8DnhAWU3N17pD9EkQh27QpzRN1zD+y/P4wfD9/7nqYvF0mx2i5P/UdGohBpiJUrw8SCS5bAJZeEuaMOOyzuqEQapRqThru/l6lAROrs88/hgQfgoYfg4IPh2Wfh0kvjjkqkUYvaEC6SXd58M9QuSkrCZahHH4VDDok7KpFGL9bWQTPra2alZrbGzAYnefw2M1ttZivN7C9m9rU44pQssn07/PSncPLJ8Omn8MILMGGCEoZIhsSWNMwsDxgNnA90BQaYWdcqxZYBBYkG96mEAYbSVL36KvTsGRq8Bw0KtYwLLog7KpEmJc6aRi9gjbu/6+5fAFOA/pULuPt8d9+e2HwD6JDhGCUbbN0KN9wAffqEEd7z54exF61axR2ZSJMTZ9JoD6yrtL0+cV91BgIvpTUiyT6zZoU5op54Am67LfSU6tMn7qhEmqw4G8KTdaD3pAXNriSsS35GNY8PAgYBdOrUKVXxSZw2b4Zbbw3tFV27wqJF0Lt33FGJNHlx1jTWAx0rbXcANlYtZGZnAz8H+rn758l25O5j3b3A3Qvatm2blmAlg6ZNC4li0iQYMgSWLlXCEMkScdY0FgNdzOwIwiSIVwDfrVzAzI4HngD6uvumzIcoGfXRR2F97qlT4YQTYPbs0PAtIlkjtpqGu+8CbgRmE6Ze/5O7l5jZUDPrlyg2HGgJPGtmy82sMKZwJZ3cYeLEULuYOTMM1isqUsIQyUKxDu5z9xeBF6vcd3el22dnPCjJrHXr4Npr4aWX4JRTwuJIxxwTd1QiUg1N/SnxqKgIPaK6dQvjL0aOhAULlDBEspymEZHMe+cduOYaeOUVOOssePJJOOKIuKMSkQhU05DMKS+HESOge3dYtiyspDd3rhKGSA5RTUMyo6QEBg4MDdwXXhhGdLevaSyniGQj1TQkvXbuhPvvD11o33knjL2YMUMJQyRHqaYh6bN0aZi+fMUKuOIKGDUKNPhSJKeppiGp99lncNdd0KsXbNoUahaTJythiDQCqmlIar3+emi7KC0NtYxHHoHWreOOSkRSRDUNSY1t2+CWW+Cb3ww1jTlzwkA9JQyRRkU1DWm4efPgRz+C994Lc0c9+CC0bBl3VCKSBqppSP1t2RIG6Z1zDjRvHkZ0jxqlhCHSiClpSP3MnBmmAHnmGbjzTli+HE47Le6oRCTNlDSkbsrK4LvfhX79oE2bMFhv2DBo0SLuyEQkA5Q0JBp3mDIlTF8+dSoMHQqLF8OJJ8YdmYhkkBrCpXYbN8L110NhYRh78fTTYd1uEWlyYq1pmFlfMys1szVmNjjJ4182sz8mHi8ys86Zj7IJc4dx40LtYs4cePjhsFa3EoZIkxVbTcPM8oDRwDmE9cIXm1mhu6+uVGwg8Im7H2VmVwC/Ar6TrpimL9vA8NmlbNyyg3atW3DHeUdz0fHR50gaMn0Vk4vWUe5OnhknHXkQ//h4x177Axg+u5QNW3aQZ0a5O+0rHWv6sg38srCELTt27tnvQfvn8+0eh/P8ig/23J/fDModKrzmmJoZfL3tAazZ9H8kK3pA8zwe+M/ue73OIdNXsWB2MffPeozT/7GMtd0KWPPAo9yxfDtbfj4r8vnIds3zDHdnZ8Xe9+cZtGqRzyfbd+7znAOa5/F/X5Tvee9at8jHDD7ZvnOf9xPY671sZuH9al+Pzxb8+/NZ+bOT7DMkkk7mXsu3TroObHYy8Et3Py+xfReAuz9UqczsRJm/mtmXgA+Btl5D0AUFBV5cXFzneKYv28Bdz61ix87yPfe1yM/joYu7R/pDHDJ9FX944/0ay+Q3MzDYWb5v+C3y87jkxPb88c117KwtE6RYXjPjkct6ctHx7fnFcyto9vjj/Per43EzHurzAyYd1xesWdKkI8nlNwtf5tW9lXX5bEHyz2dD9ylSmZktcfeC2srFeXmqPbCu0vb6xH1JyyTWFP8UOCQdwQyfXbrPH+SOneUMn10a6fmTi9bVWmZnhSdNGLuPNbko8wkDoLzCw+ssLaXfLf/FvfOeYHGHbpw7cDT/c/wFuBJGne2sqD5hQN0+W5D889nQfYrUR5wN4Zbkvqp/ZlHKYGaDgEEAnTp1qlcwG7fsqNP9VZWnoMaWin3UR15FORfOmgi/nEwXy+e2b9/Kc92+BZbs9EuqRP1s1aVsXfYpUh9xJo31QMdK2x2AjdWUWZ+4PHUgsLnqjtx9LDAWwuWp+gTTrnULNiT5g2vXOtr4g93XlhsiFfuoq2M2reXXL42kx4dr4OKLOa/df/LRAQdlNIamKupna3fZZJ/PhuxTpD7ivDy1GOhiZkeYWXPgCqCwSplC4KrE7UuBl2tqz2iIO847mhb5eXvd1yI/b0+DZm0G9O5Ya5n8ZkZ+XvL/3lvk5zGgd8fQ7pEBzXft5NaFf2Dm+J/Qbus/efPXY2DaNM45q2fS8qpz1E1+M6Omt7Iuny1I/vls6D5F6iO2pJFoo7gRmA28DfzJ3UvMbKiZ9UsUexo4xMzWALcB+3TLTZWLjm/PQxd3p33rFhihh0tdGhXvv6g7V57UibzEJZ08M079+sF77W/4ZT0ZfmlP2if+G9xddvex7r+oO8Mv60nrFvl77fug/fO58qROe92f34wav5R2a2bQ5dAD9vrS77mxlJnjb+GWRVN4qdsZFD2/kF53XFvt67jypE48+p3j9okr1zXPM/KT/AXkWTjnyRzQPC9RJpyf1i3y95St/H4Ov6wnIy7f+5ztfr/q+tmCvT+flY9V9TOkRnBJt9h6T6VLfXtPNQnbt8Pdd8Ojj0K7dvDEE3DBBXFHJSJZIGrvKY0IbypefTXMSLtmDVx7Lfz619CqVdxRiUiO0dxTjd3WrXDDDdCnD1RUwMsvw5gxShgiUi9KGo3ZrFlhyo8xY+DWW2HlSjjzzLijEpEcpstTjdHmzSFJTJgA3/hGmC/qpJPijkpEGgHVNBqbadPCBIOTJsGQIbBsmRKGiKSMahqNxUcfhfW5p06F448Pl6aOOy7uqESkkVFNI9e5w8SJoXYxcyY8+GBYTU8JQ0TSQDWNXLZuXeg++9JLcMopYXGkY46JOyoRacRU08hFFRVhYF63bmH8xciRsGCBEoaIpJ1qGrnmnXfCIL1XXoGzzoKxY+HII+OOSkSaCNU0ckV5OYwYAd27w9Kl8OSTMHeuEoZHpz1zAAAJfElEQVSIZJRqGrmgpAQGDgwN3BdeCI8/Du01MZ2IZJ5qGtls5064/3444YQwZ9SkSTBjhhKGiMRGNY1stXQp/PCHsGIFfOc7MGoUHHpo3FGJSBOnmka2+ewzuOsu6NULNm2C6dNhyhQlDBHJCqppZJPXXw9tF6WloZbx8MNwkJZeFZHsEUtNw8wONrO5Zva/id/7fDOa2XFm9lczKzGzlWb2nThizYht2+CWW+Cb3ww1jdmzw0A9JQwRyTJxXZ4aDPzF3bsAfyH5Mq7bge+7ezegL/AbM2udwRgzY9680I121Cj48Y/hrbfg3HPjjkpEJKm4kkZ/YHzi9njgoqoF3P3v7v6/idsbgU1A24xFmG5btoRBeuecA82bw8KF8Nhj0LJl3JGJiFQrrqTxVXf/ACDxu8ZWXjPrBTQH3slAbOk3c2aYAuT3v4c774Tly+G00+KOSkSkVmlrCDezecBhSR76eR33czgwEbjK3SuqKTMIGATQqVOnOkaaQWVloe1i8uRwSWrGDCiodR13EZGskbak4e5nV/eYmX1kZoe7+weJpLCpmnKtgBeAIe7+Rg3HGguMBSgoKPCGRZ4G7vDHP8JNN8Gnn8K998LgweGylIhIDonr8lQhcFXi9lXAjKoFzKw58Gdggrs/m8HYUmvjRrjoIhgwAI44Igzau/tuJQwRyUlxJY1hwDlm9r/AOYltzKzAzJ5KlLkcOB242syWJ35yZ2Uhdxg3LiyONGcODB8e1uo+9ti4IxMRqTdzz76rOQ1RUFDgxcXF8Qbxj3/AoEFhFtrTT4ennoIuXeKNSUSkBma2xN1rbWTVNCKpVFERus0eeyz89a/wu9/B/PlKGCLSaGgakVQpLQ3jLl57Dfr2DSvrZXNPLhGRelBNo6F27YJf/Qp69gzrXjzzDLz4ohKGiDRKqmk0xMqVYWLBJUvg4oth9Gg4LNnQFBGRxkE1jfr4/PPQbfbEE2HdOnj2WZg2TQlDRBo91TTq6s03Q+2ipASuvBJ+8xs45JC4oxIRyQjVNKLavh3uuANOPjmM6n7hBZg4UQlDRJoU1TSiWLAgLI60Zg1cey38+tfQqlXcUYmIZJxqGjX517/ghhvgjDPCGIyXX4YxY5QwRKTJUtKozqxZYfryMWPg1ltDT6kzz4w7KhGRWClpVLV5M1x9NZx/flgQadEiGDECDjgg7shERGKnNo3K3ngjzEj78ccwZEj4+fKX445KRCRrKGlU9vWvQ48eoaH7uNyZUFdEJFOUNCpr2zZMYy4iIkmpTUNERCJT0hARkchiSRpmdrCZzTWz/038PqiGsq3MbIOZ/TaTMYqIyL7iqmkMBv7i7l2AvyS2q3Mf8GpGohIRkRrFlTT6A+MTt8cDFyUrZGYnAl8F1DotIpIF4koaX3X3DwASvw+tWsDMmgGPAHfUtjMzG2RmxWZWXFZWlvJgRUQkSFuXWzObByRbYOLnEXdxA/Ciu68zsxoLuvtYYCxAQUGB1yVOERGJLm1Jw93Pru4xM/vIzA539w/M7HBgU5JiJwPfNLMbgJZAczPb5u41tX+IiEgamXvm/zE3s+HAx+4+zMwGAwe7+3/XUP5qoMDdb4yw7zLgvZQF23BtgH/GHUQ95XLskNvxK/Z4NOXYv+bubWsrFNeI8GHAn8xsIPA+cBmAmRUA17n7NfXdcZQXnUlmVuzuBXHHUR+5HDvkdvyKPR6KvXaxJA13/xg4K8n9xcA+CcPdnwGeSXtgIiJSI40IFxGRyJQ00m9s3AE0QC7HDrkdv2KPh2KvRSwN4SIikptU0xARkciUNFLEzPqaWamZrUl0I676+NVmVmZmyxM/9e4hlmpmNs7MNpnZW9U8bmY2KvHaVprZCZmOsToRYu9jZp9WOu93ZzrG6phZRzObb2Zvm1mJmd2SpExWnvuIsWfluTez/czsTTNbkYj93iRlvmxmf0yc9yIz65z5SPcVMfb0fte4u34a+APkAe8ARwLNgRVA1yplrgZ+G3es1cR/OnAC8FY1j18AvAQYcBJQFHfMdYi9D/B83HFWE9vhwAmJ218B/p7kc5OV5z5i7Fl57hPnsmXidj5QBJxUpcwNwJjE7SuAP8Yddx1iT+t3jWoaqdELWOPu77r7F8AUwqSMOcHdFwCbayjSH5jgwRtA68RI/thFiD1rufsH7r40cftfwNtA+yrFsvLcR4w9KyXO5bbEZn7ip2rjbuVJVacCZ1lt8xllQMTY00pJIzXaA+sqba8n+R/QJYlLDFPNrGNmQkuJqK8vW52cqM6/ZGbd4g4mmcTlj+MJ/zlWlvXnvobYIUvPvZnlmdlywhRGc9292vPu7ruAT4FDMhtlchFihzR+1yhppEay/0CqZv+ZQGd37wHM49//xeSCKK8vWy0lTI/QE3gMmB5zPPsws5bANOAn7r616sNJnpI1576W2LP23Lt7ubsfB3QAepnZsVWKZO15jxB7Wr9rlDRSYz1QOZt3ADZWLuDuH7v754nNJ4ETMxRbKtT6+rKVu2/dXZ139xeBfDNrE3NYe5hZPuFL93/c/bkkRbL23NcWe7afewB33wK8AvSt8tCe825mXwIOJMsug1YXe7q/a5Q0UmMx0MXMjjCz5oSGs8LKBapch+5HuAacKwqB7yd68pwEfOqJ9VCynZkdtvtatJn1InzmP443qiAR19PA2+4+oppiWXnuo8SerefezNqaWevE7RbA2cDfqhQrBK5K3L4UeNkTrcxxihJ7ur9r4pqwsFFx911mdiMwm9CTapy7l5jZUKDY3QuBm82sH7CL8B/L1bEFXIWZTSb0dGljZuuBewgNbLj7GOBFQi+eNcB24AfxRLqvCLFfClxvZruAHcAV2fDHn3Aq8D1gVeIaNcDPgE6Q9ec+SuzZeu4PB8abWR4hkf3J3Z+v8vf6NDDRzNYQ/l6viC/cvUSJPa3fNRoRLiIikenylIiIRKakISIikSlpiIhIZEoaIiISmZKGiIhEpqQhEgMz+0e2DXQTiUJJQ0REIlPSEEkzM5tuZksS6x8MijsekYbQiHCR9Puhu29OTPuw2MymxR2QSH0paYik381m9p+J2x2BLnEGI9IQShoiaWRmfQiTyp3s7tvN7BVgv1iDEmkAtWmIpNeBwCeJhHEMYclWkZylpCGSXrOAL5nZSuA+4I2Y4xFpEM1yKyIikammISIikSlpiIhIZEoaIiISmZKGiIhEpqQhIiKRKWmIiEhkShoiIhKZkoaIiET2/wFEQGvaWQkOKQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c309c7be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot that includes the regression line\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If **al=3**, what class do we predict for household? **1**\n",
    "\n",
    "If **al=1.5**, what class do we predict for household? **0**\n",
    "\n",
    "We predict the 0 class for **lower** values of al, and the 1 class for **higher** values of al. What's our cutoff value? Around **al=2**, because that's where the linear regression line crosses the midpoint between predicting class 0 and class 1.\n",
    "\n",
    "Therefore, we'll say that if **household_pred >= 0.5**, we predict a class of **1**, else we predict a class of **0**.\n",
    "\n",
    "## $$h_\\beta(x) = \\beta_0 + \\beta_1x_1 + \\beta_2x_2 + ... + \\beta_nx_n$$\n",
    "\n",
    "- $h_\\beta(x)$ is the response\n",
    "- $\\beta_0$ is the intercept\n",
    "- $\\beta_1$ is the coefficient for $x_1$ (the first feature)\n",
    "- $\\beta_n$ is the coefficient for $x_n$ (the nth feature)\n",
    "\n",
    "### if $h_\\beta(x)\\le 0.5$ then $\\hat y = 0$ \n",
    "\n",
    "### if $h_\\beta(x)> 0.5$ then $\\hat y = 1$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['small', 'big', 'small'], dtype='<U5')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# understanding np.where\n",
    "import numpy as np\n",
    "nums = np.array([5, 15, 8])\n",
    "\n",
    "# np.where returns the first value if the condition is True, and the second value if the condition is False\n",
    "np.where(nums > 10, 'big', 'small')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "      <th>household_pred</th>\n",
       "      <th>household_pred_class</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.340495</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.315436</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.230236</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  household_pred  household_pred_class  \n",
       "id                                                              \n",
       "22   1.521227          0       -0.340495                     0  \n",
       "185  1.521103          1       -0.315436                     0  \n",
       "40   1.520781          0       -0.250283                     0  \n",
       "39   1.520781          0       -0.250283                     0  \n",
       "51   1.520682          0       -0.230236                     0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform household_pred to 1 or 0\n",
    "glass['household_pred_class'] = np.where(glass.household_pred >= 0.5, 1, 0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGvVJREFUeJzt3XuUnHWd5/H3p6sr0AElYOIROgkBJuJGbsEeLsMclxEdLjMD6HCdZRR1J8eZRXdHlyO4LCDKyjGrM7oy62QdvC8o4mQynLgcHeUs4y6YYIQIGE+GaxJWAhJ0TDTVne/+URcrlaqupzv9q6pfz+d1Tp2q53l+9avv8+uq+vRzqSpFBGZmZgBD/S7AzMwGh0PBzMwaHApmZtbgUDAzswaHgpmZNTgUzMyswaFgZmYNDgUzM2twKJiZWcNwvwuYqvnz58eSJUv6XYaZWVYefPDB5yNiQbd22YXCkiVLWL9+fb/LMDPLiqSnirTz7iMzM2twKJiZWYNDwczMGhwKZmbW4FAwM7MGh4KZmTU4FMzMrCFZKEi6TdJzkn7YYbkkfVLSZkkPSzo5VS1mZlZMyg+vfQ74FPCFDsvPBZbWLqcC/7123TOrN2xl5T2b2LZjF0fMG+Hqs4/lwuWjA9fnoLpu9UZuf+AZJiIoSZx29KE8+cKututeZFyKjl1zu0NGykjw4s7KPu1G543wO69ZwHd+tJ2tO3YhoP6L5IfOLXPDH7x2r/7r/W7dsYuSxEQE80bK/OJXFSp79u57brn6/9TO2oLW/lprbO2jXstUamw3PkDbsajXP9o0jq33rz9up+nW8W837jt2Vgo9z1dv2MqNax5hx67KPuvWbb261TKV19lkdQyqXr+nKCK6t5pu59IS4O6IOK7Nsr8G7o2I22vTm4AzI+LZyfocGxuLmfhE8+oNW7n26xvZVZlozBspl/jIW46f9oCn6HNQXbd6I1+6/+lJ29TXHeg6LkXHrl276SqXxMqLTmy8Me1vv/X+YN/1TdFneUggqExM/hoeKZf4w9eNcteDW6dUU/P4dxufyZ7nqzds5eo7H6KyZ+86yyVx6W8u2qeuduvVrZYir7PJ6qg/DwbNTL6nSHowIsa6tutjKNwN3BIR/1ib/gfg/REx6Tv+TIXCGbd8m607du0zf3TeCN+95g0D0+egOubatUwUeO6MzhsB6DouRceuU7vpqvc/U/12Wt9X/ex5PvP1DzF399QfY3hIAIzvmf5rtXkLZKqPvfiwuTz9051dH7/ettVk951KXd1q6fT4Rerodt9+aa75ruPO4tbfuhSY3ntK0VDo53cfqc28tn8xSSuAFQCLFy+ekQff1uENoNP8fvU5qIoEAky+7s3Lio7dTI9lvb+Z6rdTP69+/imO+8k/cd+RJ/HTuYfMyGP1yuKTjmDDD7YVbtuq6H1nopZ2j1+0jsnu2y/NNT/78vmN2ynfU/oZCluARU3TC4G2f7WIWAWsguqWwkw8+BHzRtr+Z3hE7T+9QelzUNX3WXdzxCRbCs3jUnTsOrWbrnr/M9Vvp/Ud3lPd/P+vr/9jHjri2Cn1OdnWVlFF/17tHvuCa97ARwtsSdXbtprsvlOpq1stnR6/SB3d7tsvnWpO+Z7Sz1NS1wBvrZ2FdBrwUrfjCTPp6rOPZaRc2mveSLnUOMg1KH0OqstPXdS1TX3di4xL0bFr1266yiU1+p+Jfuv9teurvGccgPHS1P4Pm7TPIVEutdvg3ttIucTlpy6a8vo1j3+38ZnseX712cdWjxO0KJfUtq5269WtliKvs8nqGNTXaD/eU5JtKUi6HTgTmC9pC3ADUAaIiE8Da4HzgM3ATuDtqWppp36QZiaP6qfoc1B9+MLqAeSiZx/B5ONSdOxa283U2UfN/c7E2UetNR4Q1S2FylD1BT6ds4/ajU+nsWh39tHYkYdN++yjTuNe5Oyj+vxOZ/201tXt7KPpvs661TGI+vGekvRAcwozdaDZrKe+/GW44gr40Y/g2MH8r9Rmt6IHmv2JZrNeqNS2ZMrl/tZh1oVDwawXHAqWCYeCWS84FCwTDgWzXnAoWCYcCma94FCwTDgUzHrBoWCZcCiY9YJDwTLhUDDrhUoFJCjNzKexzVJxKJj1QqVS3UpQ96+lMOsnh4JZL9RDwWzAORTMesGhYJlwKJj1gkPBMuFQMOsFh4JlwqFg1gsOBcuEQ8GsFxwKlgmHglkvOBQsEw4Fs15wKFgmHApmveBQsEw4FMx6waFgmXAomPWCQ8Ey4VAw6wWHgmXCoWDWCw4Fy4RDwawXHAqWCYeCWS/s3u1QsCw4FMx6wVsKlgmHglkvOBQsEw4Fs15wKFgmHApmveBQsEw4FMx6waFgmXAomPWCQ8EykTQUJJ0jaZOkzZKuabN8saTvSNog6WFJ56Wsx6xvHAqWiWShIKkE3AqcCywDLpe0rKXZdcBXI2I5cBnwV6nqMesrh4JlIuWWwinA5oh4PCJ2A3cAF7S0CeDltduHANsS1mPWHxEwMeFQsCykDIVR4Jmm6S21ec1uBK6QtAVYC7y7XUeSVkhaL2n99u3bU9Rqlk6lUr12KFgGUoaC2syLlunLgc9FxELgPOCLkvapKSJWRcRYRIwtWLAgQalmCTkULCMpQ2ELsKhpeiH77h56J/BVgIj4v8CBwPyENZn1nkPBMpIyFNYBSyUdJWkO1QPJa1raPA2cBSDpX1ENBe8fstnFoWAZSRYKETEOXAXcAzxG9SyjRyTdJOn8WrP3AX8i6SHgduDKiGjdxWSWN4eCZWQ4ZecRsZbqAeTmedc33X4UOCNlDWZ951CwjPgTzWapORQsIw4Fs9QcCpYRh4JZag4Fy4hDwSw1h4JlxKFglppDwTLiUDBLzaFgGXEomKXmULCMOBTMUnMoWEYcCmapORQsIw4Fs9QcCpYRh4JZag4Fy4hDwSw1h4JlxKFglppDwTLiUDBLzaFgGXEomKXmULCMOBTMUnMoWEYcCmapORQsIw4Fs9QcCpYRh4JZavVQmDOnv3WYFeBQMEutUgEJSqV+V2LWlUPBLLVKxbuOLBsOBbPUHAqWEYeCWWoOBcuIQ8EsNYeCZcShYJaaQ8Ey4lAwS82hYBlxKJil5lCwjDgUzFJzKFhGHApmqTkULCPDky2UtBGITssj4oQu9z8H+ARQAj4TEbe0aXMJcGPtcR6KiD/qXrZZRhwKlpFJQwH4/dr1v6tdf7F2/W+AnZPdUVIJuBV4E7AFWCdpTUQ82tRmKXAtcEZEvCjplVOs32zwORQsI5OGQkQ8BSDpjIg4o2nRNZK+C9w0yd1PATZHxOO1Pu4ALgAebWrzJ8CtEfFi7fGem/oqmA04h4JlpOgxhYMk/XZ9QtJvAQd1uc8o8EzT9JbavGavBl4t6buS7q/tbjKbXRwKlpFuu4/q3gncJumQ2vQO4B1d7qM281qPTwwDS4EzgYXAfZKOi4gde3UkrQBWACxevLhgyWYDolKBgw/udxVmhRQKhYh4EDhR0ssBRcRLBe62BVjUNL0Q2Namzf0RUQGekLSJakisa3n8VcAqgLGxsY4Hvs0GkrcULCPdzj56b4f5AETExye5+zpgqaSjgK3AZUDrmUWrgcuBz0maT3V30uOFKjfLhUPBMtJtS+Fl0+04IsYlXQXcQ/WU1Nsi4hFJNwHrI2JNbdnvSnoUmACujogXpvuYZgPJoWAZ6Xb20Qf3p/OIWAusbZl3fdPtAN5bu5jNTg4Fy0ihs48kLZT0t5Kek/QTSXdJWpi6OLNZwaFgGSl6SupngTXAEVRPK/372jwz68ahYBkpGgoLIuKzETFeu3wOWJCwLrPZw6FgGSkaCs9LukJSqXa5AvABYbMiHAqWkaKh8A7gEuD/Ac8CF9H9w2tmBg4Fy0rRD689DZyfuBaz2cmhYBkpFAqSFlD98rolzfeJCG8tmE0mAsbHHQqWjaLfffR3wH3At6h+yMzMihgfr147FCwTRUNhbkS8P2klZrNRpVK9dihYJooeaL5b0nlJKzGbjRwKlpluX4j3c6pfdy3gA5J2A7tr0xERL09folnGHAqWmW7ffTTtL8QzMxwKlp2i332k2ofX/nNtepGkU9KWZjYLOBQsM0WPKfwVcDq//j2EfwZuTVKR2Wyye3f12qFgmSh69tGpEXGypA0AEfGipDkJ6zKbHbylYJkpuqVQkVSi9hvLtQ+z7UlWldls4VCwzBQNhU8Cfwu8UtLNwD8C/yVZVWazhUPBMlP0u4++LOlB4Cyqp6NeGBGPJa3MbDZwKFhmip59dAzwRETcCvwQeJOkeUkrM5sNHAqWmaK7j+4CJiT9BvAZ4Cjgfyarymy2cChYZoqGwp6IGAfeAnwiIv4cODxdWWazhEPBMjOVs48uB94K3F2b52e5WTcOBctM0VB4O9UPr90cEU9IOgr4UrqyzGYJh4JlpujZR48C72mafgK4JVVRZrOGQ8EyU/SX156g9sG1ZhFx9IxXZDabOBQsM0W/5mKs6faBwMXAYTNfjtks41CwzBQ6phARLzRdtkbEXwJvSFybWf4cCpaZoruPTm6aHKK65eDfWjDrxqFgmSm6++hjTbfHgSeBS2a8GrPZxqFgmSl69tHvpC7EbFZyKFhmin730SGSPi5pfe3yMUmHFLjfOZI2Sdos6ZpJ2l0kKSSNdWpjlqV6KMzxz49YHop+eO024OdUdxldAvwM+Oxkd6j9/sKtwLnAMuByScvatHsZ1c9APFC8bLNMeEvBMlM0FI6JiBsi4vHa5YNAt88onAJsrrXfDdwBXNCm3YeAjwK/LFy1WS4qFZCgVOp3JWaFFA2FXZJ+uz4h6QxgV5f7jALPNE1vqc1rkLQcWBQRdzMJSSvqu662b99esGSzAVCpeCvBslL07KM/BT7fdBzhReBtXe6jNvMan4qWNAT8BXBltwePiFXAKoCxsbF9PlltNrAcCpaZoqHwGNVdPMcA84CXgAuBhye5zxZgUdP0QmBb0/TLgOOAeyUBvApYI+n8iFhfsC6zweZQsMwUDYW/A3YA3we2FrzPOmBp7RtVtwKXAX9UXxgRLwHz69OS7gX+owPBZhWHgmWmaCgsjIhzptJxRIxLugq4BygBt0XEI5JuAtZHxJop1mqWH4eCZaZoKPwfScdHxMapdB4Ra4G1LfOu79D2zKn0bZYFh4JlZtJQkLSR6sHhYeDtkh4HfkX1IHJExAnpSzTLmEPBMtNtS+H3e1KF2WzlULDMTBoKEfFUrwoxm5UcCpaZoh9eM7PpcChYZhwKZik5FCwzDgWzlBwKlhmHgllKDgXLjEPBLCWHgmXGoWCWkkPBMuNQMEvJoWCZcSiYpeRQsMw4FMxScihYZhwKZik5FCwzDgWzlBwKlhmHgllKDgXLjEPBLCWHgmXGoWCWkkPBMuNQMEvJoWCZcSiYpRIB4+MOBcuKQ8EslfHx6rVDwTLiUDBLpVKpXjsULCMOBbNUHAqWIYeCWSoOBcuQQ8EsFYeCZcihYJaKQ8Ey5FAwS8WhYBlyKJil4lCwDDkUzFJxKFiGHApmqTgULENJQ0HSOZI2Sdos6Zo2y98r6VFJD0v6B0lHpqzHrKccCpahZKEgqQTcCpwLLAMul7SspdkGYCwiTgC+Bnw0VT1mPedQsAyl3FI4BdgcEY9HxG7gDuCC5gYR8Z2I2FmbvB9YmLAes95yKFiGUobCKPBM0/SW2rxO3gl8I2E9Zr3lULAMDSfsW23mRduG0hXAGPCvOyxfAawAWLx48UzVZ5aWQ8EylHJLYQuwqGl6IbCttZGkNwL/CTg/In7VrqOIWBURYxExtmDBgiTFms04h4JlKGUorAOWSjpK0hzgMmBNcwNJy4G/phoIzyWsxaz3HAqWoWShEBHjwFXAPcBjwFcj4hFJN0k6v9ZsJXAwcKekH0ha06E7s/w4FCxDKY8pEBFrgbUt865vuv3GlI9v1lcOBcuQP9FslopDwTLkUDBLxaFgGXIomKVSD4U5c/pbh9kUOBTMUtm9u3rtLQXLiEPBLBXvPrIMORTMUnEoWIYcCmapVCogQanU70rMCnMomKVSqXgrwbLjUDBLxaFgGXIomKXiULAMORTMUnEoWIYcCmapOBQsQw4Fs1QcCpYhh4JZKg4Fy5BDwSwVh4JlyKFglopDwTLkUDBLxaFgGXIomKXiULAMORTMUnEoWIYcCmapOBQsQw4Fs1QcCpYhh4JZKg4Fy5BDwSwVh4JlyKFglopDwTLkUDBLxaFgGXIomKXiULAMORTMUnEoWIYcCmapOBQsQw4Fs1QcCpYhh4JZKg4Fy9Bwys4lnQN8AigBn4mIW1qWHwB8AXgd8AJwaUQ8maKW1Ru2svKeTWzbsYsj5o1w9dnHcuHy0Sn1cd3qjdz+wDNMRFCSOO3oQ3nyhV179Qlw45pH2LGrAsChc8vc8AevbTzW6g1b91peb/N7JxzO3Q8925hfHoLKnn1rmFseojKxp7FsSHDMgoPY/NwviDY1HzSnxM1vPn6vdW1dj8tPXcTYkYftU1fu5pTE7ol9R+XQuWVe3FmhJDERv14uYO6cEr/YPdFYNm+kzO7xCXbWBrz579n6txwS7AkYrT+/phgK9efo1h27Go9fvx6d5nPWbKoU0e6tZAY6lkrAj4E3AVuAdcDlEfFoU5s/A06IiHdJugx4c0RcOlm/Y2NjsX79+inVsnrDVq79+kZ2VSYa80bKJT7yluMLv8iuW72RL93/9KRtykPVF/CeliEtl8TKi04E4Oo7H6LS2iCx0pD42MUncuHy0Y7rIWgbKravcklc+puL+Mr3nun4txwZHuKxm8+D66+HD36wa5/tnqP79DnF56xZM0kPRsRYt3YptxROATZHxOO1gu4ALgAebWpzAXBj7fbXgE9JUsxwUq28ZxO7KhOctfkBlv3k8cb87feX4fVHF+rjFd/8Me/ej7K2f+9OAN71y/78J7593Z3w+qP3ez2saug+8a5JxnGovqzglkL9OTqZXZUJVt6zyaFgSaUMhVHgmabpLcCpndpExLikl4BXAM83N5K0AlgBsHjx4ikXsm3HLgB+98f3c+nGb+698FvF+vjzKT/qAPrWLFmPTExoiNJrXlOobf05OlPtzKYrZSiozbzWf62KtCEiVgGroLr7aKqFHDFvhK07dnHNue/m2nOuaswfnTfCfe9/Q6E+ln5g7V77n6dqdN4IAFv79KKur+v+rodVtR6PaGd03gj3XfSmQv3Vn6NF2pmllPLsoy3AoqbphcC2Tm0kDQOHAD+d6UKuPvtYRsolQkPsGSqxZ6jEAQfM4X3nLoNSqdDl0tOXNO7b6VIaHoZSm/nlYd537jLed+4ySsPDXfuZ6YuGhxvr2mk9osc15XwplYe59PQlk/4tG8+vKT5HJzNSLjVOZjBLJWUorAOWSjpK0hzgMmBNS5s1wNtqty8Cvj3TxxMALlw+ykfecjyj80YQ1f/gpnrA7sMXHs8Vpy2mpOrGTUnijGMO26vPlRefyMcvOYl5I7/ej3zo3DIrL6oe5L1w+SgrLz5xr+X1Nlectniv+eUOf5m55aG9lg0Jlr7yoLabXFA9+6h+kLnTelxx2mL+4tKT9qkrd3NK7Ufl0LnV9ayPQZ2ojlfzsnkjZeY2DXj97/nhC4/f5285VOtuOs+v5udo8+PXr6fTp9l0JDv7CEDSecBfUj0l9baIuFnSTcD6iFgj6UDgi8ByqlsIl9UPTHcynbOPzMz+pRuEs4+IiLXA2pZ51zfd/iVwccoazMysOH+i2czMGhwKZmbW4FAwM7MGh4KZmTU4FMzMrMGhYGZmDQ4FMzNrSPrhtRQkbQee6ncdTebT8gV+GXHt/ZFz7ZB3/f+Saz8yIhZ0a5RdKAwaSeuLfEpwELn2/si5dsi7ftfenXcfmZlZg0PBzMwaHAr7b1W/C9gPrr0/cq4d8q7ftXfhYwpmZtbgLQUzM2twKBQk6RxJmyRtlnRNm+VXStou6Qe1y7/tR52tJN0m6TlJP+ywXJI+WVuvhyWd3OsaOylQ+5mSXmoa8+vbtesHSYskfUfSY5IekfTv27QZyLEvWPsgj/2Bkr4n6aFa/R9s0+YASV+pjf0Dkpb0vtJ9Faw97XtNRPjS5UL1R4L+CTgamAM8BCxraXMl8Kl+19qm9tcDJwM/7LD8POAbVH947DTggX7XPIXazwTu7nedHWo7HDi5dvtlwI/bPGcGcuwL1j7IYy/g4NrtMvAAcFpLmz8DPl27fRnwlX7XPYXak77XeEuhmFOAzRHxeETsBu4ALuhzTYVExP9m8t+9vgD4QlTdD8yTdHhvqptcgdoHVkQ8GxHfr93+OfAY0PpbmgM59gVrH1i18fzn2mS5dmk9eHoB8Pna7a8BZ0nq9Ku2PVOw9qQcCsWMAs80TW+h/YvkD2u7Ab4maVFvSttvRddtUJ1e29T+hqTX9ruYdmq7JpZT/a+v2cCP/SS1wwCPvaSSpB8AzwHfjIiOYx8R48BLwCt6W2V7BWqHhO81DoVi2v0H0Zrefw8siYgTgG/x6/9CBl2RdRtU36f60f0Tgf8GrO5zPfuQdDBwF/AfIuJnrYvb3GVgxr5L7QM99hExEREnAQuBUyQd19JkYMe+QO1J32scCsVsAZrTeCGwrblBRLwQEb+qTf4P4HU9qm1/dV23QRURP6tvakf198DLkub3uawGSWWqb6pfjoivt2kysGPfrfZBH/u6iNgB3Auc07KoMfaShoFDGLBdlZ1qT/1e41AoZh2wVNJRkuZQPTC1prlBy77g86nuh83BGuCttTNhTgNeiohn+11UEZJeVd8PLOkUqs/nF/pbVVWtrr8BHouIj3doNpBjX6T2AR/7BZLm1W6PAG8EftTSbA3wttrti4BvR+0obj8VqT31e83wTHY2W0XEuKSrgHuonol0W0Q8IukmYH1ErAHeI+l8YJzqfxxX9q3gJpJup3qmyHxJW4AbqB68IiI+DaylehbMZmAn8Pb+VLqvArVfBPyppHFgF3DZILywa84A/hjYWNs/DPABYDEM/NgXqX2Qx/5w4POSSlTD6qsRcXfL6/VvgC9K2kz19XpZ/8rdS5Hak77X+BPNZmbW4N1HZmbW4FAwM7MGh4KZmTU4FMzMrMGhYGZmDQ4FswQkPTmIH+Yy68ahYGZmDQ4Fs/0kabWkB2vff7+i3/WY7Q9/otls/70jIn5a+1qCdZLu6ndBZtPlUDDbf++R9Oba7UXA0n4WY7Y/HApm+0HSmVS/tOz0iNgp6V7gwL4WZbYffEzBbP8cArxYC4TXUP1ZTbNsORTM9s//AoYlPQx8CLi/z/WY7Rd/S6qZmTV4S8HMzBocCmZm1uBQMDOzBoeCmZk1OBTMzKzBoWBmZg0OBTMza3AomJlZw/8HM6hWgtLDMYIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c3095ff98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$h_\\beta(x)$ can be lower 0 or higher than 1, which is countra intuitive\n",
    "\n",
    "## Using Logistic Regression Instead\n",
    "\n",
    "Logistic regression can do what we just did:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fit a logistic regression model and store the class predictions\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "logreg.fit(X, y)\n",
    "glass['household_pred_class'] = logreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGyBJREFUeJzt3X+03HV95/Hn686dwA0iARNP4SYxQCPdyA+DdyFKj0tFy4+2JFp+JJUq6jan7aLd6nIEl0VEXTlmtdWVrqYWf3ahIpqmnCirVc6qu2CCESLQ0CwgJKESkaA1UXJv3vvH/HAymbnz/X7v/Wbmfng9zplzZ77zmc+853O/M6/5fj/fmVFEYGZmBjDU7wLMzGxwOBTMzKzJoWBmZk0OBTMza3IomJlZk0PBzMyaHApmZtbkUDAzsyaHgpmZNQ33u4C85s6dG4sWLep3GWZmM8o999zz44iY16vdjAuFRYsWsWnTpn6XYWY2o0j6YZZ23n1kZmZNDgUzM2tyKJiZWZNDwczMmhwKZmbW5FAwM7Mmh4KZmTWVFgqSbpL0pKQfdLlekj4qaZuk+ySdXlYtZmaWTZkfXvs08DHgs12uPx9YXD+dCfyP+t9DZt3mHay5Yys7d+/luDkjXHnuSaxYOjpwfQ6qa9Zt4ea7H2cigorEshOO5tGn9nZ87FnGJevYtbY7aqSKBE/v2XdQu9E5I/zWb8zjm/+0ix279yKg8YvkR8+u8u7fe8kB/Tf63bF7LxWJiQjmjFT5+S/3sW//gX3PrtbeT+2pX9HeX3uN7X00aslTY6fxATqORaP+0ZZxbL994367XW4f/07jvnvPvkzr+brNO7hu/f3s3rvvoMfW63H1qiXP82yyOgbVoX5NUUT0blW0c2kRcHtEnNzhuk8Ad0bEzfXLW4GzI+KJyfocGxuL6fhE87rNO7j6S1vYu2+iuWykWuEDrzul8ICX0eegumbdFj5/12OTtmk8dqDnuGQdu07tiqpWxJqLTmu+ME2130Z/cPDjLaPP6pBAsG9i8ufwSLXC779slNvu2ZGrptbx7zU+k63n6zbv4Mpb72Xf/gPrrFbEpf92wUF1dXpcvWrJ8jybrI7GejBopvM1RdI9ETHWs10fQ+F24IaI+Hb98j8C74yISV/xpysUzrrhG+zYvfeg5aNzRvjOVa8amD4H1YlXb2Aiw7ozOmcEoOe4ZB27bu2KavRfpN/fefBbvP3bn0ct4zA8JADG90/f82o6+mzdAsl73wuPmc1jP9nT8/4bbdtNdts8dfWqpdv9Z6mj1237pbXm204+hxtfcSlQ7DUlayj087uP1GFZx/+YpNXAaoCFCxdOy53v7PIC0G15v/ocVFkCASZ/7K3XZR276R7LRn9F+l32+BbmP/MkX33xK6a1pkGz8KXHsfn7OzO3bZf1ttNRS6f7z1rHZLftl9aan3j+3Ob5Ml9T+hkK24EFLZfnAx3/axGxFlgLtS2F6bjz4+aMdHxneFz9ne2g9DmoGvusezluki2F1nHJOnbd2hXV6L9Iv5X9E+weOZI/u/DK5rLJtoyKmo4+s/6/Ot338qtexQczbEk12rab7LZ56upVS7f7z1JHr9v2S7eay3xN6echqeuBN9SPQloGPNNrPmE6XXnuSYxUKwcsG6lWmpNcg9LnoFp15oKebRqPPcu4ZB27Tu2KqlbU7L9Iv5X9+5nQr55Cjf7KqLFTn9UhUa102uA+0Ei1wqozF+SuqXX8ez2mydbzK889qTZP0KZaUce6Oj2uXrVkeZ5NVsegPkf78ZpS2paCpJuBs4G5krYD7waqABHxcWADcAGwDdgDvKmsWjppTNJM56x+GX0OqvetqE0gZz36CCYfl6xj195uuo4+au0369FHh7GfiaFKx/7aa5yOo486jU+3seh09NHYi44pfPRRt3HPcvRRY3m3o37a6+p19FHR51mvOgZRP15TSp1oLsN0TTSbTdnrXw/f/S788z/3uxKznrJONPsTzWZFjY/D8Iz7nSqzSTkUzIoaH4fK9MwdmA0Kh4JZURMT3lKw5DgUzIry7iNLkEPBrCiHgiXIoWBWlOcULEEOBbOiPKdgCXIomBXl3UeWIIeCWVEOBUuQQ8GsKM8pWIIcCmZFeU7BEuRQMCvKu48sQQ4Fs6IcCpYgh4JZUQ4FS5BDwawoTzRbghwKZkV5otkS5FAwK8q7jyxBDgWzohwKliCHgllRnlOwBDkUzIrynIIlyKFgVpR3H1mCHApmRTkULEEOBbOiPKdgCXIomBUR4TkFS5JDwayI/ftrfx0KlhiHglkR4+O1vw4FS4xDwayIRih4TsES41AwK2JiovbXWwqWGIeCWRHefWSJciiYFeFQsESVGgqSzpO0VdI2SVd1uH6hpG9K2izpPkkXlFmP2bRxKFiiSgsFSRXgRuB8YAmwStKStmbXAF+IiKXASuCvyqrHbFp5otkSVeaWwhnAtoh4OCKeBW4Blre1CeD59fNHATtLrMds+nii2RJVZiiMAo+3XN5eX9bqOuAySduBDcBbO3UkabWkTZI27dq1q4xazfLx7iNLVJmhoA7Lou3yKuDTETEfuAD4nKSDaoqItRExFhFj8+bNK6FUs5wcCpaoMkNhO7Cg5fJ8Dt499BbgCwAR8X+Bw4G5JdZkNj08p2CJKjMUNgKLJR0vaRa1ieT1bW0eA84BkPRvqIWC9w/Z4POcgiWqtFCIiHHgCuAO4EFqRxndL+l6SRfWm70D+CNJ9wI3A5dHRPsuJrPB491HlqhS1+iI2EBtArl12bUt5x8AziqzBrNSOBQsUf5Es1kRnlOwRDkUzIrwnIIlyqFgVoR3H1miHApmRTgULFEOBbMiPKdgiXIomBXhLQVLlEPBrAhPNFuiHApmRXhLwRLlUDArwnMKliiHglkR3lKwRDkUzIrwnIIlyqFgVoS3FCxRDgWzIhwKliiHglkRnmi2RDkUzIrwnIIlyqFgVoR3H1miHApmRTgULFEOBbMiPKdgiXIomBXhULBEORTMipiYgKGh2sksIV6jzYoYH/d8giXJoWBWxPi4dx1ZkhwKZkV4S8ES5VAwK2JiwqFgSXIomBXhLQVLlEPBrAjPKViiHApmRXhLwRLlUDArwnMKliiHglkR3lKwRE26VkvaAkS36yPi1B63Pw/4CFABPhkRN3RocwlwXf1+7o2IP+hdtlmfeU7BEtXrrc7v1v/+h/rfz9X/vh7YM9kNJVWAG4HXANuBjZLWR8QDLW0WA1cDZ0XE05JemLN+s/7wloIlatK1OiJ+CCDprIg4q+WqqyR9B7h+kpufAWyLiIfrfdwCLAceaGnzR8CNEfF0/f6ezP8QzPrAoWCJyjqncISk32xckPQK4IgetxkFHm+5vL2+rNWLgRdL+o6ku+q7m8wGnyeaLVFZ1+q3ADdJOqp+eTfw5h63UYdl7fMTw8Bi4GxgPvAtSSdHxO4DOpJWA6sBFi5cmLFksxJ5S8ESlWmtjoh7gNMkPR9QRDyT4WbbgQUtl+cDOzu0uSsi9gGPSNpKLSQ2tt3/WmAtwNjYWNeJb7NDxhPNlqheRx+9vctyACLiw5PcfCOwWNLxwA5gJdB+ZNE6YBXwaUlzqe1OejhT5Wb95C0FS1SvtfrIoh1HxLikK4A7qB2SelNE3C/pemBTRKyvX/fbkh4AJoArI+KpovdpdshMTMCsWf2uwmza9Tr66D1T6TwiNgAb2pZd23I+gLfXT2Yzx/g4zJ7d7yrMpl2mo48kzZf0ZUlPSvqRpNskzS+7OLOB5TkFS1TWQ1I/BawHjqN2WOk/1JeZPTd5TsESlTUU5kXEpyJivH76NDCvxLrMBps/p2CJyhoKP5Z0maRK/XQZ4Alhe+7yloIlKmsovBm4BPgX4AngInp/eM0sXZ5TsERl/fDaY8CFJddiNnN4S8ESlWmtljSP2pfXLWq9TUR4a8GemzynYInKulb/PfAt4OvUPmRm9tzmLQVLVNa1enZEvLPUSsxmEs8pWKKyTjTfLumCUisxm0m8pWCJ6vWFeD+j9nXXAt4l6Vng2frliIjnl1+i2QByKFiien33UeEvxDNLmieaLVFZv/tI9Q+v/Zf65QWSzii3NLMB5i0FS1TWOYW/Al7Or34P4V+BG0upyGwm8ESzJSrrW50zI+J0SZsBIuJpSf4yeXtuivDuI0tW1i2FfZIq1H9juf5htv2lVWU2yPbXV32HgiUoayh8FPgy8EJJ7we+DfzX0qoyG2Tj47W/DgVLUNbvPvpbSfcA51A7HHVFRDxYamVmg6oRCp5TsARlPfroROCRiLgR+AHwGklzSq3MbFB5S8ESlnX30W3AhKRfBz4JHA/8z9KqMhtkE/Wv/3IoWIKyhsL+iBgHXgd8JCL+HDi2vLLMBpi3FCxheY4+WgW8Abi9vqxaTklmA85zCpawrKHwJmofXnt/RDwi6Xjg8+WVZTbAvKVgCct69NEDwNtaLj8C3FBWUWYDzaFgCcv6y2uPUP/gWquIOGHaKzIbdJ5otoRlXavHWs4fDlwMHDP95ZjNAJ5TsIRlmlOIiKdaTjsi4i+BV5Vcm9lg8u4jS1jW3Uent1wcorbl4N9asOcmh4IlLOta/aGW8+PAo8Al016N2UzgOQVLWNajj36r7ELMZgzPKVjCsn730VGSPixpU/30IUlHZbjdeZK2Stom6apJ2l0kKSSNdWtjNjC8+8gSlvXDazcBP6O2y+gS4KfApya7Qf33F24EzgeWAKskLenQ7khqn4G4O3vZZn3kULCEZQ2FEyPi3RHxcP30HqDXZxTOALbV2z8L3AIs79DuvcAHgV9krtqsnzynYAnLGgp7Jf1m44Kks4C9PW4zCjzecnl7fVmTpKXAgoi4nUlIWt3YdbVr166MJZuVxFsKlrCsa/WfAJ9pmUd4Gnhjj9uow7Lmp6IlDQF/AVze684jYi2wFmBsbOygT1abHVKeaLaEZQ2FB6nt4jkRmAM8A6wA7pvkNtuBBS2X5wM7Wy4fCZwM3CkJ4NeA9ZIujIhNGesyO/S8pWAJy7pW/z2wG/gesCPjbTYCi+vfqLoDWAn8QePKiHgGmNu4LOlO4D85EGzgORQsYVnX6vkRcV6ejiNiXNIVwB1ABbgpIu6XdD2wKSLW56zVbDB4otkSlnWt/j+STomILXk6j4gNwIa2Zdd2aXt2nr7N+sZzCpawSUNB0hZqk8PDwJskPQz8ktokckTEqeWXaDZgvPvIEtZrrf7dQ1KF2UziULCETbpWR8QPD1UhZjOG5xQsYVk/vGZmDZ5TsIQ5FMzy8u4jS5hDwSwvh4IlzKFglpfnFCxhDgWzvDynYAlzKJjl5d1HljCHglle3lKwhDkUzPIaH4ehodrJLDFeq83ympjwVoIly6Fgltf4uOcTLFkOBbO8HAqWMIeCWV4OBUuYQ8Esr4kJh4Ily6Fgltf4uCeaLVkOBbO8vPvIEuZQMMvLoWAJcyiY5eU5BUuYQ8EsL88pWMIcCmZ5efeRJcyhYJaXQ8ES5lAwy8tzCpYwh4JZXp5TsIQ5FMzy8u4jS5hDwSwvh4IlzKFglpdDwRLmUDDLyz+yYwkrNRQknSdpq6Rtkq7qcP3bJT0g6T5J/yjpRWXWYzYtvKVgCSstFCRVgBuB84ElwCpJS9qabQbGIuJU4IvAB8uqx2zaOBQsYWVuKZwBbIuIhyPiWeAWYHlrg4j4ZkTsqV+8C5hfYj1m08OhYAkrMxRGgcdbLm+vL+vmLcBXSqzHbHp4TsESVubbHXVYFh0bSpcBY8C/63L9amA1wMKFC6erPrNivKVgCStzS2E7sKDl8nxgZ3sjSa8G/jNwYUT8slNHEbE2IsYiYmzevHmlFGuWmUPBElZmKGwEFks6XtIsYCWwvrWBpKXAJ6gFwpMl1mI2fRwKlrDSQiEixoErgDuAB4EvRMT9kq6XdGG92RrgecCtkr4vaX2X7swGh78QzxJW6podERuADW3Lrm05/+oy79+sFP5CPEuYP9Fslpd3H1nCHApmeTkULGEOBbO8HAqWMIeCWV7+8JolzKFglpe3FCxhDgWzPCJ8SKolzaFglsfERO2vQ8ES5VAwy6MRCp5TsEQ5FMzyGB+v/fWWgiXKoWCWh0PBEudQMMvDoWCJcyiY5eE5BUucQ8EsD28pWOIcCmZ5OBQscQ4FszwcCpY4h4JZHg4FS5xDwSwPTzRb4hwKZnl4S8ES51Awy8OhYIlzKJjl4VCwxDkUzPLwnIIlzqFgloe3FCxxDgWzPBwKljiHglkeDgVLnEPBLA/PKVjiHApmeXhLwRLnUDDLw6FgiXMomOXhULDEORTM8miEgucULFEOBbM8GhPN3lKwRJW6Zks6D/gIUAE+GRE3tF1/GPBZ4GXAU8ClEfFoGbWs27yDNXdsZefuvRw3Z4Qrzz2JFUtHc/Vxzbot3Hz340xEUJFYdsLRPPrU3gP6BLhu/f3s3rsPgKNnV3n3772keV/rNu844PpGm9859Vhuv/eJ5vLqEOzbf3ANs6tD7JvY37xuSHDivCPY9uTPiQ41HzGrwvtfe8oBj7X9caw6cwFjLzrmoLpmulkV8ezEwaNy9OwqT+/ZR0ViIn51vYDZsyr8/NmJ5nVzRqo8Oz7BnvqA/+FDm3kvwPDwQf/LIcH+gNGC61djHd2xe2/z/ht/i/ZplpciOr2UTEPHUgV4CHgNsB3YCKyKiAda2vwpcGpE/LGklcBrI+LSyfodGxuLTZs25apl3eYdXP2lLezdN9FcNlKt8IHXnZL5SXbNui18/q7HJm1THao9gfe3DWm1ItZcdBoAV956L/vaG5SsMiQ+dPFprFg62vVxCDqGih3o4vv+F2u+8lH+2ye+yicenej6v8y7fnVaR6fap1krSfdExFivdmVuKZwBbIuIh+sF3QIsBx5oabMcuK5+/ovAxyQppjmp1tyxlb37Jjhn290s+dHDzeW77qrCK0/I1McLvvYQb51CWbu+eysAf/yL/rwT37XxVnjlCVN+HM91p/7LNgC+fO+P2HfkC7q227tvgjV3bM38At5YRyeTt0+zIsoMhVHg8ZbL24Ezu7WJiHFJzwAvAH7c2kjSamA1wMKFC3MXsnP3XgB++6G7uHTL1w688uvZ+vjz3Pc6gL6eyOPos11HzOGpw4/o2a6x3mWRtW2ePs2KKDMU1GFZ+1vULG2IiLXAWqjtPspbyHFzRtixey9Xnf9Wrj7viuby0TkjfOudr8rUx+J3bThg/3Neo3NGANjRpyd147FO9XEYhMTQUAV6jONx9f95Fo11NEs7szKVefTRdmBBy+X5wM5ubSQNA0cBP5nuQq489yRGqhVCQ+wfqrB/qMJhh83iHecvqR1amOF06csXNW/b7VQZHoZKh+XVYd5x/hLecf4SKsPDPfuZ7pOGh5uPtdvjiENc00w+DQ9XWHXmAqpDnd7T1IxUK80DD/Kso5PJ26dZEWWGwkZgsaTjJc0CVgLr29qsB95YP38R8I3pnk8AWLF0lA+87hRG54wgau+a807YvW/FKVy2bCEV1V4IKhJnnXjMAX2uufg0PnzJS5kzUm3e7ujZVdZcVJvkXbF0lDUXn3bA9Y02ly1beMDyapf/zOzq0AHXDQkWv/CIjptcUDv6qDHJ3O1xXLZsIX9x6UsPqmumm1XpPCpHz649zsYYNIjaeLVeN2ekyuyWAW/8P9+34pSD/peNjCiyfrWuo6333/hbpE+zIko7+ghA0gXAX1I7JPWmiHi/pOuBTRGxXtLhwOeApdS2EFY2Jqa7KXL0kZnZc90gHH1ERGwANrQtu7bl/C+Ai8uswczMsvMnms3MrMmhYGZmTQ4FMzNrciiYmVmTQ8HMzJocCmZm1uRQMDOzplI/vFYGSbuAH/a7jhZzafsCvxnEtffHTK4dZnb9z+XaXxQR83o1mnGhMGgkbcryKcFB5Nr7YybXDjO7ftfem3cfmZlZk0PBzMyaHApTt7bfBUyBa++PmVw7zOz6XXsPnlMwM7MmbymYmVmTQyEjSedJ2ippm6SrOlx/uaRdkr5fP/37ftTZTtJNkp6U9IMu10vSR+uP6z5Jpx/qGrvJUPvZkp5pGfNrO7XrB0kLJH1T0oOS7pf0Zx3aDOTYZ6x9kMf+cEnflXRvvf73dGhzmKS/q4/93ZIWHfpKD5ax9nJfayLCpx4naj8S9P+AE4BZwL3AkrY2lwMf63etHWp/JXA68IMu118AfIXaD48tA+7ud805aj8buL3fdXap7Vjg9Pr5I4GHOqwzAzn2GWsf5LEX8Lz6+SpwN7Csrc2fAh+vn18J/F2/685Re6mvNd5SyOYMYFtEPBwRzwK3AMv7XFMmEfG/mfx3r5cDn42au4A5ko49NNVNLkPtAysinoiI79XP/wx4EGj/Lc2BHPuMtQ+s+nj+a/1itX5qnzxdDnymfv6LwDmSuv/o9iGSsfZSORSyGQUeb7m8nc5Pkt+v7wb4oqQFh6a0Kcv62AbVy+ub2l+R9JJ+F9NJfdfEUmrv+loN/NhPUjsM8NhLqkj6PvAk8LWI6Dr2ETEOPAO84NBW2VmG2qHE1xqHQjad3kG0p/c/AIsi4lTg6/zqXcigy/LYBtX3qH10/zTgvwPr+lzPQSQ9D7gN+I8R8dP2qzvcZGDGvkftAz32ETERES8F5gNnSDq5rcnAjn2G2kt9rXEoZLMdaE3j+cDO1gYR8VRE/LJ+8a+Blx2i2qaq52MbVBHx08amdtR+D7wqaW6fy2qSVKX2ovq3EfGlDk0Gdux71T7oY98QEbuBO4Hz2q5qjr2kYeAoBmxXZbfay36tcShksxFYLOl4SbOoTUytb23Qti/4Qmr7YWeC9cAb6kfCLAOeiYgn+l1UFpJ+rbEfWNIZ1Nbnp/pbVU29rr8BHoyID3dpNpBjn6X2AR/7eZLm1M+PAK8G/qmt2XrgjfXzFwHfiPosbj9lqb3s15rh6ewsVRExLukK4A5qRyLdFBH3S7oe2BQR64G3SboQGKf2juPyvhXcQtLN1I4UmStpO/BuapNXRMTHgQ3UjoLZBuwB3tSfSg+WofaLgD+RNA7sBVYOwhO77izgD4Et9f3DAO8CFsLAj32W2gd57I8FPiOpQi2svhARt7c9X/8G+JykbdSeryv7V+4BstRe6muNP9FsZmZN3n1kZmZNDgUzM2tyKJiZWZNDwczMmhwKZmbW5FAwK4GkRwfxw1xmvTgUzMysyaFgNkWS1km6p/7996v7XY/ZVPgTzWZT9+aI+En9awk2Srqt3wWZFeVQMJu6t0l6bf38AmBxP4sxmwqHgtkUSDqb2peWvTwi9ki6Ezi8r0WZTYHnFMym5ijg6Xog/Aa1n9U0m7EcCmZT81VgWNJ9wHuBu/pcj9mU+FtSzcysyVsKZmbW5FAwM7Mmh4KZmTU5FMzMrMmhYGZmTQ4FMzNrciiYmVmTQ8HMzJr+P8K3V8Vpx6aoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c308c5da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted the **predicted probabilities** instead of just the **class predictions**, to understand how confident we are in a given prediction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# store the predicted probabilites of class 1\n",
    "glass['household_pred_prob'] = logreg.predict_proba(X)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYU/W9x/H3l0wGBhRGBIqyCFrUuiNTN+oVe1XUR0WsCyht3WrV2l6tUvXqFUqrqNQFra1FtG69KC6lSGm5LrRWWq0gKqilIm4MKogMIDMw2/f+cTIhDJmZM0vmJJnP63nyTE5ycvLJmSTf/M7vnN8xd0dERASgU9QBREQke6goiIhIkoqCiIgkqSiIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgkFUQdoLl69erlgwYNijqGiEhOWbRo0efu3rup+XKuKAwaNIiFCxdGHUNEJKeY2Ydh5tPmIxERSVJREBGRJBUFERFJUlEQEZEkFQUREUlSURARkSQVBRERScpYUTCzB8xstZktbeB+M7O7zGy5mb1pZgdnKouIiISTyYPXHgR+CTzcwP0nAEMSl0OBXyf+tptZi0uZMm8Zq8oq2LW4iPEj9+LUof2ybpnZ6vpZS5jxysfUuBMz47Ddd+KDtRVpX3uY9RJ23aXO16MojhmsK6/abr5+xUUcvXdv5v9rDaVlFRhQd0bynbrGmXDyvtssv265pWUVxMyocae4KM6mLVVU1W677K7x4PdUeeKO+surn7H+MuqyNCdjuvUDpF0Xdfn7pazH+o+ve96Gpuuv/3Trvay8KtT7fNbiUibOfouyiqrtXltTr6upLM35nDWWI1u193eKuXvTc7V04WaDgDnuvl+a+34D/MXdZySmlwEj3P2TxpZZUlLibXFE86zFpVz79BIqqmqStxXFY0w+bf8Wr/BMLDNbXT9rCY++/FGj89S9dqDJ9RJ23aWbr6XiMWPK6Qcmv5hau9y65cH2rzcTy4x3MjCoqmn8M1wUj/GtYf14alFpszKlrv+m1k9j7/NZi0sZ/8QbVNVumzMeM876+oDtcqV7XU1lCfM5ayxH3fsg27Tld4qZLXL3kibni7AozAFudveXEtPPA1e7e6Pf+G1VFIbf/AKlZRXb3d6vuIgF13wza5aZrfa4di41Id47/YqLAJpcL2HXXUPztVTd8ttquY293iiXWddyaMlzh10/Db3PG3tsc3I1laWpz1ljOVr8GXWH6mrYsgUqK9v87+x/vk9VxWYKq6v46+7DeHL/Y1qcN2xRiHLsI0tzW9p3h5ldBFwEMHDgwDZ58lUNvDkauj2qZWarsB/kxl576n1h111br8u65bXVcjPxv26LZbakIKQ+d5gMLXn/NydXU1nS3l5dDV98AWvWMODNf3JA+Xp2rtjATuXrKd78JZ2rK4nXVNO5pgpeu7dlX95trbAwuHTuzKFbnMpYnMpYAe/0Gdz4a20jURaFlcCAlOn+wKp0M7r7NGAaBC2FtnjyXYuL0v5q2DXxqyxblpmtwv7C27WRX7mp6yXsumtovpaqW35bLbex1xvlMlvaUmjO+mnofd7YY5uTq275/bp3pqq0lIFln9Jvwxp2Ll9Pz/L19K8ph9HTYc0a+Pzz4O+6dcGveeCxesv7srCIzQWFVHUqoLawEGo/hc6dk1/IdO4M3btvnU69vf5tbfU3Hgfb+nv5tAZaN5n8TomyKMwGLjOzxwg6mNc31Z/QlsaP3Cvttrq6Tq5sWWa2GnvogFB9CnWvvan1EnbdpZuvpeIxSy6/LZaburxMZIyiTyHs+mnsfT5+5F7N7lMoqK2m3+elDF63ioFlnzJ4w2cc02UTPPQD/rrifWKV2/5Cr+oUo6bnzlDRB3r3hgMOCP727g29ekHv3ry0Dm5e+DmfdelOWdGOVMXiyRxTTj+QflnYpxDFd0rGioKZzQBGAL3MbCUwAYgDuPu9wFzgRGA5UA6cl6ks6dR10rRlr34mlpmtfn5q0IEcdu8jaHy9hF139edrq72PUpfbFnsf1c/YFnsfpVs/Da2LdHsflezWs8V7HzW03sPsfVR3e0N7/RzWs4DnHnqGXd57m6FlKzm0fBU7fPAesarK5DKqdtiR+JCvwr77Ejv5ZF6P9+ShTzrxZqwH8V37cvGoYZx6cP+0z1/nG8CFib2PqnJk76MovlMy2tGcCW3V0SwiEaithaVL4eWX4ZVXgr/vvJPcxMOAAbD//sFlv/1g771h991hp5222awizZcLHc0i0hFs3Ahz5sBTT8H8+UHHLwSbdQ47DMaODf6WlEBxcbRZRUVBRDJg06agEMycCXPnwubNsMsuMGoUjBgBw4cHLQD9+s86Kgoi0jbKy4MCMHNmUBAqKqBvX7jwQjjrLDjiCOik4daynYqCiLTc5s3wpz8FheCZZ4IWQp8+cO65QSH4xjcgFos6pTSDioKINN+//w133gmPPhr0GfTqBePGwZlnwn/8BxToqyVX6T8nIuEtXgwTJgStgsLCoJP4nHPg6KNVCPKE/osi0rSVK+G66+CRR4LdQ2+4AS69FL7ylaiTSRtTURCRhm3cCLfcArfdFhxLMH48XHutdh3NYyoKIrK96mqYPj3YVLR6NZx9Ntx4IwwaFHUyyTAVBRHZyh3++Ef4yU+CI42PPDLYvfTrX486mbQT7TQsIoHSUjjhBDj5ZKipgVmz4K9/VUHoYNRSEBF4/HG45JLg/AB33hl0IsfjUaeSCKilINKRlZUFu5SOGQN77gmvvw7/9V8qCB2YioJIR7V0aTAI3cyZMGkSvPQSDBkSdSqJmDYfiXREM2fCeedBjx5Bv8ERR0SdSLKEWgoiHUlNDVx9dTAu0UEHwaJFKgiyDbUURDqKykr49reDVsIllwQdyoWFUaeSLKOiINIRbN4MZ5wRHHMwZQpcdVXUiSRLqSiI5LsvvwxObjN/Ptx7L3z/+1EnkiymoiCSz8rK4MQT4Z//hIcfDoa3FmmEioJIvlqzBo47Dt56C554AkaPjjqR5AAVBZF8VFoKxx4LH3wQnPtg5MioE0mOUFEQyTerV8OIEfDZZ/DnPwdnQhMJSUVBJJ9s2gQnnRS0FJ5/Hg4/POpEkmNUFETyRXV1cHrMRYvg979XQZAWUVEQyQfu8KMfBf0H99wDp5wSdSLJURrmQiQf3HIL/PrXwclxLr006jSSw1QURHLdY48F500eMwYmT446jeQ4FQWRXLZkCZx/fnDazAcfhE76SEvr6B0kkqs2bIDTTw+Gv545Ezp3jjqR5IGMFgUzO97MlpnZcjO7Js39A81svpktNrM3zezETOYRyRvucOGF8N57weajvn2jTiR5ImNFwcxiwD3ACcA+wFgz26febNcDM919KDAG+FWm8ojklbvvDoauuOkmOOqoqNNIHslkS+EQYLm7r3D3SuAxYFS9eRzonrjeA1iVwTwi+eEf/4ArrwxGPh0/Puo0kmcyWRT6AR+nTK9M3JZqIjDOzFYCc4EfpluQmV1kZgvNbOGaNWsykVUkN3z+OZx5JgwcGHQsm0WdSPJMJotCuner15seCzzo7v2BE4FHzGy7TO4+zd1L3L2kd+/eGYgqkgPcg3MhrF4NTz4JxcVRJ5I8lMkjmlcCA1Km+7P95qELgOMB3P0fZtYF6AWszmAukdw0YwY8/XRwoNrQoVGnkTyVyZbCq8AQMxtsZoUEHcmz683zEfCfAGb2NaALoO1DIvWtWgU/+EEwntGVV0adRvJYxoqCu1cDlwHzgHcI9jJ6y8wmmVndwCxXAt8zszeAGcC57l5/E5NIx+YO3/sebNkCDz0EsVjUiSSPZXRAPHefS9CBnHrbDSnX3waGZzKDSM574AGYOxfuuguGDIk6jeQ5HdEsks0++AAuvxyOPjrYfCSSYSoKItmqtjYY1wiC1oLGNZJ2oPMpiGSrhx6C+fPhN7+BQYOiTiMdhH56iGSjtWuDo5WHDw/GOBJpJyoKItno6quhrCw4cY42G0k70rtNJNssWAD33w8//jHsv3/UaaSDUVEQySZVVXDxxTBgANxwQ9Pzi7QxdTSLZJOpU2HpUpg1C3bYIeo00gGppSCSLT79FH76UzjppGBYbJEIqCiIZIsJE2DzZrj99qiTSAemoiCSDZYsgenTg6OWNZSFREhFQSQbXHUV9OihzmWJnDqaRaL25z/D//1fsNmoZ8+o00gHp5aCSJSqq4PzI3z1qxrwTrKCWgoiUZo+Hd5+G556CgoLo04jopaCSGQ2bAj6EI48EkaPjjqNCKCWgkh0Jk+GNWuCE+iYRZ1GBFBLQSQaH34Id9wB48ZBSUnUaUSSVBREonD99UHr4Kabok4isg0VBZH2tmQJ/O538MMfBgPfiWQRFQWR9vY//wM77hicM0Eky6goiLSnV16BP/whOKvazjtHnUZkOyoKIu3phhugVy+4/PKok4ikpaIg0l7+/vdgOIvx43WuBMlaKgoi7WXiROjdW8NZSFbTwWsi7WHBAnj2WZgyBbp1izqNSIPUUhBpDxMnQp8+cMklUScRaZRaCiKZ9tJL8Nxz8ItfqJUgWU8tBZFMUytBckijLQUzWwJ4Q/e7+wFNPP54YCoQA6a7+81p5jkTmJh4njfc/eymY4vkiL/9DZ5/Hm67Dbp2jTqNSJOa2nx0UuJv3e4SjyT+ngOUN/ZAM4sB9wDHAiuBV81stru/nTLPEOBaYLi7rzOzPs3ML5LdJk6Er3wFLr446iQioTRaFNz9QwAzG+7uw1PuusbMFgCTGnn4IcByd1+RWMZjwCjg7ZR5vgfc4+7rEs+3uvkvQSRLvfgivPBCcJpNtRIkR4TtU+hmZt+omzCzI4Cmesz6AR+nTK9M3JZqT2BPM1tgZi8nNjeJ5IeJE6FvX7USJKeE3fvoAuABM+uRmC4Dzm/iMenOGlK/f6IAGAKMAPoDfzOz/dy9bJsFmV0EXAQwcODAkJFFIrRgAcyfH5wzoago6jQioYUqCu6+CDjQzLoD5u7rQzxsJZA6LnB/YFWaeV529yrgfTNbRlAkXq33/NOAaQAlJSUNdnyLZI3Jk4MB7773vaiTiDRLU3sf/biB2wFw99sbefirwBAzGwyUAmOA+nsWzQLGAg+aWS+CzUkrQiUXyVZvvgl//CP87Gc6LkFyTlMthR1bumB3rzazy4B5BLukPuDub5nZJGChu89O3Hecmb0N1ADj3X1tS59TJCvcfHMw4J3GOJIcZO65tTWmpKTEFy5cGHUMkfSWL4e99oIrr4Rbb406jUiSmS1y9yZPCB5q7yMz629mvzez1Wb2mZk9ZWb9Wx9TJM9MmQLxOFxxRdRJRFok7C6pvwVmA7sS7Fb6TOI2EamzahU8+CCcdx7sskvUaURaJGxR6O3uv3X36sTlQaB3BnOJ5J477oDq6uAkOiI5KmxR+NzMxplZLHEZB6hDWKTOF1/AvffCmDGw++5RpxFpsbBF4XzgTOBT4BPgdJo+eE2k4/jlL+HLL+Gaa6JOItIqYQ9e+wg4JcNZRHLTpk1w111w8smw//5RpxFplVBFwcx6EwxeNyj1Me6u1oLIfffB2rVw7bVRJxFptbBjH/0B+BvwHMFBZiICUFkZnFHtqKPg8MOjTiPSamGLQld3vzqjSURy0SOPQGkpPPBA1ElE2kTYjuY5ZnZiRpOI5JqaGrjlFjj4YDj22KjTiLSJpgbE20gw3LUB/21mlUBlYtrdvXvmI4pkqaefhnffhSeeAEs3UrxI7mnqzGstHhBPJK+5B8Nj77knjB4ddRqRNhN27CNLHLz2P4npAWZ2SGajiWSxefNg8eLguIRYLOo0Im0mbJ/Cr4DD2Xo+hC+BezKSSCQXTJ4M/fvDOedEnUSkTYXd++hQdz/YzBYDuPs6MyvMYC6R7PX3v8OLL8Kdd0KhPgaSX8K2FKrMLEbiHMuJg9lqM5ZKJJvVnWrzwgujTiLS5sIWhbuA3wN9zOxG4CXgpoylEslWb74Jc+bA5ZfrVJuSl8KOffQ7M1sE/CfB7qinuvs7GU0mko10qk3Jc2H3PtoDeN/d7wGWAseaWXFGk4lkm/feg8cfh0sugZ12ijqNSEaE3Xz0FFBjZl8FpgODgf/NWCqRbKRTbUoHELYo1Lp7NXAaMNXdrwB0vkHpOD75BH77W51qU/Jec/Y+Ggt8B5iTuC2emUgiWei224KxjnSqTclzYYvCeQQHr93o7u+b2WDg0czFEskia9cGp9ocO1an2pS8F3bvo7eBH6VMvw/cnKlQIlll6tTg7Go6iY50AGHPvPY+iQPXUrm7fjZJftuwAe6+Oxj0bp99ok4jknFhh7koSbneBTgD6Nn2cUSyzK9+BWVlcN11UScRaReh+hTcfW3KpdTd7wS+meFsItEqL4fbb4eRI2HYsKjTiLSLsJuPDk6Z7ETQctC5FiS/3X8/rFmjVoJ0KGE3H92Wcr0a+AA4s83TiGSLykq49VY48sjgItJBhN376OhMBxHJKo88AitXwvTpUScRaVdhxz7qYWa3m9nCxOU2M+sR4nHHm9kyM1tuZtc0Mt/pZuZmVtLQPCLtpro6GPhu2DA47rio04i0q7AHrz0AbCTYZHQmsAH4bWMPSJx/4R7gBGAfYKyZbbdPn5ntSHAMxCvhY4tk0BNPwPLlQV+CWdRpRNpV2KKwh7tPcPcVictPgaaOUTgEWJ6YvxJ4DBiVZr6fAbcCm0OnFsmU2lq46abgmIRR6d6uIvktbFGoMLNv1E2Y2XCgoonH9AM+TplembgtycyGAgPcfQ6NMLOL6jZdrVmzJmRkkRZ45hlYujQ4erlT2I+HSP4Iu/fRJcBDKf0I64DvNvGYdO3u5FHRZtYJuAM4t6knd/dpwDSAkpKS7Y6sFmkT7nDjjcH4RmPGRJ1GJBJhi8I7BJt49gCKgfXAqcCbjTxmJTAgZbo/sCplekdgP+AvFmy37QvMNrNT3H1hyFwibWfuXHj1VbjvPigI+9EQyS9h3/l/AMqA14DSkI95FRiSGFG1FBgDnF13p7uvB3rVTZvZX4CrVBAkEu4wcSIMHgzfbaoRLJK/whaF/u5+fHMW7O7VZnYZMA+IAQ+4+1tmNglY6O6zm5lVJHPmzIGFC4OjmOM6VYh0XObe9CZ6M5sG3O3uSzIfqXElJSW+cKEaE9KG3KGkJBj47l//UlGQvGRmi9y9yWPBGm0pmNkSgs7hAuA8M1sBbCHoRHZ3P6AtwopEavZseO01ePBBFQTp8JrafHRSu6QQiUptLUyYAEOGwDnnRJ1GJHKNFgV3/7C9gohEYtYseOMNePhh7XEkQviD10TyT21tsMfRnnsG518WkdB7H4nkn6efhiVL4NFH1UoQSVBLQTqm6mq4/nrYe28dvSySQj+PpGN66CFYtixoLcRiUacRyRpqKUjHU1ER7HF02GFw6qlRpxHJKmopSMfzy19CaSn87nc6X4JIPWopSMdSVgaTJ8MJJ8BRR0WdRiTrqChIx3LLLVsLg4hsR0VBOo5Vq2DqVDj7bDjwwKjTiGQlFQXpOK6/PtgVddKkqJOIZC0VBekYFi0KBry7/PLgzGoikpaKguQ/d7jiCujVC667Luo0IllNu6RK/nvySfjb3+A3v4EePZqeX6QDU0tB8tvmzTB+PBxwAFxwQdRpRLKeWgqS3+64Az78EF54QcNZiISgloLkr1Wr4KabgqEsjj466jQiOUFFQfLXFVdAVRX84hdRJxHJGSoKkp/mzYOZM4O9jfbYI+o0IjlDRUHyT0UF/OAHwRnVfvKTqNOI5BR1NEv+mTwZ3nsPnn8eOneOOo1ITlFLQfLLsmXBoHfnnAPf/GbUaURyjoqC5A93uPRSKCqC226LOo1ITtLmI8kf06YFxyPcey985StRpxHJSWopSH54/3248ko45hi46KKo04jkLBUFyX21tcEQFp06wfTpOsWmSCto85Hkvl//GubPh/vug912izqNSE7LaEvBzI43s2VmttzMrklz/4/N7G0ze9PMnjczfaKleVasCI5FGDlSA96JtIGMFQUziwH3ACcA+wBjzWyferMtBkrc/QDgSeDWTOWRPFRdDd/5DsTj2mwk0kYy2VI4BFju7ivcvRJ4DBiVOoO7z3f38sTky0D/DOaRfDNhAixYEGw+6q+3jkhbyGRR6Ad8nDK9MnFbQy4A/pTBPJJPnn02OHL5ggtg7Nio04jkjUx2NKdry3vaGc3GASXAUQ3cfxFwEcDAgQPbKp/kqk8/hXHj4Gtfg7vuijqNSF7JZEthJTAgZbo/sKr+TGZ2DHAdcIq7b0m3IHef5u4l7l7Su3fvjISVHFFbC9/+NmzcGIyC2rVr1IlE8komi8KrwBAzG2xmhcAYYHbqDGY2FPgNQUFYncEski8mTYLnngtaCPvuG3UakbyTsaLg7tXAZcA84B1gpru/ZWaTzOyUxGxTgB2AJ8zsdTOb3cDiROCpp+CnP4Xvfle7n4pkiLmn3cyftUpKSnzhwoVRx5D29sYbcMQRcMABwYFqXbpEnUgkp5jZIncvaWo+DXMh2W/1ajjlFOjZE37/exUEkQzSMBeS3Sor4VvfCgrDSy9B375RJxLJayoKkr1qaoI9jV56CWbMgGHDok4kkve0+Uiykztcdlmw2+mUKTBmTNSJRDoEFQXJThMmBCfLufpquOqqqNOIdBgqCpJ9pk6Fn/0s2O108uSo04h0KCoKkl3uuQcuvxxGjw5aChr5VKRdqShI9rjzzqAfYdSooGO5QPtBiLQ3FQXJDrfeCldcEex++sQT0Llz1IlEOiQVBYlWbW3QkXz11cEeRjNmBCfNEZFIqH0u0dm8OehM/t//DTYbTZ0KnfQ7RSRKKgoSjU8/DTqTX34ZbroJrrlGncoiWUBFQdrf4sVBZ/LatcHIp6edFnUiEUlQW13ajztMmwaHHx5cf+klFQSRLKOiIO1j48ZgHKPvfx9GjIDXXoOhQ6NOJSL1qChI5v31r8F5EGbMgJ//HObOBZ1WVSQrqShI5mzaBD/+MRx9NMRi8OKLcN112sNIJIvp0yltzx2efhq+9jW44w64+GJ4/XUYPjzqZCLSBBUFaVvLl8OJJwZHJhcXB62DX/0Kdtgh6mQiEoKKgrSNTz4JDkDbZx9YsCBoIbz2Ghx5ZNTJRKQZdJyCtM4XXwTjFt11V3DqzAsvhBtugF13jTqZiLSAioK0zPvvB6Oa3n8/lJfD2WfDxInw1a9GnUxEWkFFQcJzDzYN3X03PPlksBfR2WfD+PGw335RpxORNqCiIE377DN4+OGgVbBsGXTvHoxs+sMfQv/+UacTkTakoiDpffEFzJoFM2fC889DdXWwS+k118AZZ0C3blEnFJEMUFGQgDssXQrPPgvz5sELLwSFYPfd4cor4dxzYe+9o04pIhmmotCRffIJPPdcUAieey6YBthrr+BI5DPPhIMP1pDWIh2IikJHUVYWHDewaNHWv+++G9y3885wzDFw3HFw7LEwYEC0WUUkMioK+aa2FkpL4V//2vrlv2gRrFixdZ6BA4MWwPnnB4XgoIM0HpGIACoKuccd1q2DDz+Ejz7a9u+77waX8vKt8w8eDMOGBQeVDRsWDFetEUpFpAEZLQpmdjwwFYgB09395nr3dwYeBoYBa4Gz3P2DTGSZtbiUKfOWsaqsgl2Lixg/ci9OHdqvWcu4ftYSZrzyMTXuxMw4bPed+GBtxTbLBJg4+y3KKqoA2KlrnAkn75t8rlmLS7e5H3d2idcyao8d+cdrK2D9enpWbKBv+TqKN62n16Yydi4vo9emMvp8uY5+G9fQrbJim1xbCgop7d6bD4v7smKfY1nRsx8revbn7T6Dqe5RzI2j99/mtdZ/HWMPHUDJbj23zZUHCmNGZY1vd/tOXeOsK68iZkaNb73fgK6FMTZV1iTvKy6KU1ldQ3lVbfKxdf/P+v/LTga1Dv1a+P6qe4+WllUkn7/ub0uXKdJc5r79h6ZNFmwWA/4NHAusBF4Fxrr72ynzXAoc4O4Xm9kYYLS7n9XYcktKSnzhwoXNyjJrcSnXPr2Eiqqa5G1F8RiTT9u/8Q+ZO9TUQFUVk2a9wVMvf0C8tpp4TTVdqivpUr2FLlWVdKmupHN1Jd1qKyms2kJh4rYu1VvYYUsFPSo3MaJvZ+JfbuSj9z+h25ZydtyyiR0Tf+O1NQ1G+LKwiLVde/B512I+71ZMafc+lPboQ2n33pR278Oq7r1Z27VHo53BsU7GbWccyKlD+3H9rCU8+vJH281jQGbeCfknHjPO+voAHv/nx1TVpl9rod5fKdK9R1u7TJFUZrbI3UuanC+DReFwYKK7j0xMXwvg7pNT5pmXmOcfZlYAfAr09kZCtaQoDL/5BUrLKrj0HzM57t2XKaitoaCmmiKrZbcd41BVFVyqq7der7u0Uo11YmPnrlR06caXXbrxRUERGzt3ZWPnbnzZuWvy+sbO3diQuL6uqDufdyvm86492Bzv0uoMEPx6XXDNN9nj2rnb/DqWlqnfykinbp2HUfcebUpzlimSKmxRyOTmo37AxynTK4FDG5rH3avNbD2wM/B56kxmdhFwEcDAgQObHWRV4sO2JRZnfZcdqOoUozpWQHWnAnYbNhDi8eBSULD1euqloIAb/2851bEYVZ0KqIoVsLmgkM0FndlSUMiWgsJgOl53W5zNBZ3ZXFBIebwLmFH3Oz6qr+O6daCC0DbCrMdVIb7kmztvc5Yp0hKZLArptmfU/ySFmQd3nwZMg6Cl0NwguxYXUVpWwf2HjOb+Q0Ynb+9XXMRJIX91PbChdb+wdy0uAgj1azAT6p4/zC9caVqY9Vi3zsOoe4+GmU8kkzK5H+JKIHWH9/7AqobmSWw+6gF80dZBxo/ci6J4bJvbiuKxZMdwGGMPbXrf/Xgno1OaMhePGeNH7sX4kXsRTzdDhsU6WfK1NvQ6dHhaePFY0Dnf2P+yue+vdO/R1i5TpCUyWRReBYaY2WAzKwTGALPrzTMb+G7i+unAC431J7TUqUP7Mfm0/elXXIQRtBCa22H381P3Z9xhA4klOnRjZgzfo+c2y5xyxoHcfuZBFBfFk4/bqWucKacHnbynDu3HlDMO3Ob+unnGHTZwm9vjDfw3zwQbAAAFZUlEQVRnusY7bXNfJ4Mhfbo1+KXerTCW7GRu6HWMO2wgd5x10Ha5cl1hLP1a2alr8Dpj9TrnjWB9pd5XXBSna8oKr/t//vzU/bf7X9bViJa8v1Lfo6nPX/e3JcsUaYmMdTQDmNmJwJ0Eu6Q+4O43mtkkYKG7zzazLsAjwFCCFsIYd1/R8BJb1tEsItLRZUNHM+4+F5hb77YbUq5vBs7IZAYREQlPYxuIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgkqSiIiEhSRg9eywQzWwN8GHWOFL2oN4BfDlH2aORydsjt/B05+27u3uQZtnKuKGQbM1sY5ijBbKTs0cjl7JDb+ZW9adp8JCIiSSoKIiKSpKLQetOiDtAKyh6NXM4OuZ1f2ZugPgUREUlSS0FERJJUFEIys+PNbJmZLTeza9Lcf66ZrTGz1xOXC6PIWZ+ZPWBmq81saQP3m5ndlXhdb5rZwe2dsSEhso8ws/Up6/yGdPNFwcwGmNl8M3vHzN4ys/9KM09WrvuQ2bN53Xcxs3+a2RuJ/D9NM09nM3s8se5fMbNB7Z90eyGzZ/a7xt11aeJCcJKg94DdgULgDWCfevOcC/wy6qxpsv8HcDCwtIH7TwT+RHDiscOAV6LO3IzsI4A5UedsINsuwMGJ6zsC/07znsnKdR8yezavewN2SFyPA68Ah9Wb51Lg3sT1McDjUeduRvaMfteopRDOIcByd1/h7pXAY8CoiDOF4u4v0vh5r0cBD3vgZaDYzHZpn3SNC5E9a7n7J+7+WuL6RuAdoP65NLNy3YfMnrUS6/PLxGQ8canfeToKeChx/UngP80s8lOVh8yeUSoK4fQDPk6ZXkn6D8m3EpsBnjSzAe0TrdXCvrZsdXiiqf0nM9s36jDpJDZNDCX41Zcq69d9I9khi9e9mcXM7HVgNfCsuze47t29GlgP7Ny+KdMLkR0y+F2johBOul8Q9av3M8Agdz8AeI6tv0KyXZjXlq1eIzh0/0DgbmBWxHm2Y2Y7AE8Bl7v7hvp3p3lI1qz7JrJn9bp39xp3PwjoDxxiZvvVmyVr132I7Bn9rlFRCGclkFqN+wOrUmdw97XuviUxeR8wrJ2ytVaTry1bufuGuqa2B+cDj5tZr4hjJZlZnOBL9Xfu/nSaWbJ23TeVPdvXfR13LwP+Ahxf767kujezAqAHWbapsqHsmf6uUVEI51VgiJkNNrNCgo6p2akz1NsWfArBdthcMBv4TmJPmMOA9e7+SdShwjCzvnXbgc3sEIL389poUwUSue4H3nH32xuYLSvXfZjsWb7ue5tZceJ6EXAM8K96s80Gvpu4fjrwgid6caMUJnumv2sK2nJh+crdq83sMmAewZ5ID7j7W2Y2CVjo7rOBH5nZKUA1wS+OcyMLnMLMZhDsKdLLzFYCEwg6r3D3e4G5BHvBLAfKgfOiSbq9ENlPBy4xs2qgAhiTDR/shOHAt4Elie3DAP8NDISsX/dhsmfzut8FeMjMYgTFaqa7z6n3eb0feMTMlhN8XsdEF3cbYbJn9LtGRzSLiEiSNh+JiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCSAaY2QfZeDCXSFNUFEREJElFQaSVzGyWmS1KjH9/UdR5RFpDRzSLtN757v5FYliCV83sqagDibSUioJI6/3IzEYnrg8AhkQZRqQ1VBREWsHMRhAMWna4u5eb2V+ALpGGEmkF9SmItE4PYF2iIOxNcFpNkZyloiDSOn8GCszsTeBnwMsR5xFpFY2SKiIiSWopiIhIkoqCiIgkqSiIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgk/T88gTJFdrKNdAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c308b3518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.97161726 0.02838274]]\n",
      "[[0.34361555 0.65638445]]\n",
      "[[0.00794192 0.99205808]]\n"
     ]
    }
   ],
   "source": [
    "# examine some example predictions\n",
    "print(logreg.predict_proba(1))\n",
    "print(logreg.predict_proba(2))\n",
    "print(logreg.predict_proba(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column indicates the predicted probability of **class 0**, and the second column indicates the predicted probability of **class 1**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probability, odds, e, log, log-odds\n",
    "\n",
    "$$probability = \\frac {one\\ outcome} {all\\ outcomes}$$\n",
    "\n",
    "$$odds = \\frac {one\\ outcome} {all\\ other\\ outcomes}$$\n",
    "\n",
    "Examples:\n",
    "\n",
    "- Dice roll of 1: probability = 1/6, odds = 1/5\n",
    "- Even dice roll: probability = 3/6, odds = 3/3 = 1\n",
    "- Dice roll less than 5: probability = 4/6, odds = 4/2 = 2\n",
    "\n",
    "$$odds = \\frac {probability} {1 - probability}$$\n",
    "\n",
    "$$probability = \\frac {odds} {1 + odds}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds\n",
       "0         0.10  0.111111\n",
       "1         0.20  0.250000\n",
       "2         0.25  0.333333\n",
       "3         0.50  1.000000\n",
       "4         0.60  1.500000\n",
       "5         0.80  4.000000\n",
       "6         0.90  9.000000"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a table of probability versus odds\n",
    "table = pd.DataFrame({'probability':[0.1, 0.2, 0.25, 0.5, 0.6, 0.8, 0.9]})\n",
    "table['odds'] = table.probability/(1 - table.probability)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is **e**? It is the base rate of growth shared by all continually growing processes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.718281828459045"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# exponential function: e^1\n",
    "np.exp(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is a **(natural) log**? It gives you the time needed to reach a certain level of growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.999896315728952"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time needed to grow 1 unit to 2.718 units\n",
    "np.log(2.718)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also the **inverse** of the exponential function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log(np.exp(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "      <th>logodds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>-2.197225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>-1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>-1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.405465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2.197225</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds   logodds\n",
       "0         0.10  0.111111 -2.197225\n",
       "1         0.20  0.250000 -1.386294\n",
       "2         0.25  0.333333 -1.098612\n",
       "3         0.50  1.000000  0.000000\n",
       "4         0.60  1.500000  0.405465\n",
       "5         0.80  4.000000  1.386294\n",
       "6         0.90  9.000000  2.197225"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add log-odds to the table\n",
    "table['logodds'] = np.log(table.odds)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  What is Logistic Regression?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear regression:** continuous response is modeled as a linear combination of the features:\n",
    "\n",
    "$$y = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "**Logistic regression:** log-odds of a categorical response being \"true\" (1) is modeled as a linear combination of the features:\n",
    "\n",
    "$$\\log \\left({p\\over 1-p}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "This is called the **logit function**.\n",
    "\n",
    "Probability is sometimes written as pi:\n",
    "\n",
    "$$\\log \\left({\\pi\\over 1-\\pi}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "The equation can be rearranged into the **logistic function**:\n",
    "\n",
    "$$\\pi = \\frac{e^{\\beta_0 + \\beta_1x}} {1 + e^{\\beta_0 + \\beta_1x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In other words:\n",
    "\n",
    "- Logistic regression outputs the **probabilities of a specific class**\n",
    "- Those probabilities can be converted into **class predictions**\n",
    "\n",
    "The **logistic function** has some nice properties:\n",
    "\n",
    "- Takes on an \"s\" shape\n",
    "- Output is bounded by 0 and 1\n",
    "\n",
    "We have covered how this works for **binary classification problems** (two response classes). But what about **multi-class classification problems** (more than two response classes)?\n",
    "\n",
    "- Most common solution for classification models is **\"one-vs-all\"** (also known as **\"one-vs-rest\"**): decompose the problem into multiple binary classification problems\n",
    "- **Multinomial logistic regression** can solve this as a single problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Interpreting Logistic Regression Coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYU/W9x/H3l0wGBhRGBIqyCFrUuiNTN+oVe1XUR0WsCyht3WrV2l6tUvXqFUqrqNQFra1FtG69KC6lSGm5LrRWWq0gKqilIm4MKogMIDMw2/f+cTIhDJmZM0vmJJnP63nyTE5ycvLJmSTf/M7vnN8xd0dERASgU9QBREQke6goiIhIkoqCiIgkqSiIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgkFUQdoLl69erlgwYNijqGiEhOWbRo0efu3rup+XKuKAwaNIiFCxdGHUNEJKeY2Ydh5tPmIxERSVJREBGRJBUFERFJUlEQEZEkFQUREUlSURARkSQVBRERScpYUTCzB8xstZktbeB+M7O7zGy5mb1pZgdnKouIiISTyYPXHgR+CTzcwP0nAEMSl0OBXyf+tptZi0uZMm8Zq8oq2LW4iPEj9+LUof2ybpnZ6vpZS5jxysfUuBMz47Ddd+KDtRVpX3uY9RJ23aXO16MojhmsK6/abr5+xUUcvXdv5v9rDaVlFRhQd0bynbrGmXDyvtssv265pWUVxMyocae4KM6mLVVU1W677K7x4PdUeeKO+surn7H+MuqyNCdjuvUDpF0Xdfn7pazH+o+ve96Gpuuv/3Trvay8KtT7fNbiUibOfouyiqrtXltTr6upLM35nDWWI1u193eKuXvTc7V04WaDgDnuvl+a+34D/MXdZySmlwEj3P2TxpZZUlLibXFE86zFpVz79BIqqmqStxXFY0w+bf8Wr/BMLDNbXT9rCY++/FGj89S9dqDJ9RJ23aWbr6XiMWPK6Qcmv5hau9y65cH2rzcTy4x3MjCoqmn8M1wUj/GtYf14alFpszKlrv+m1k9j7/NZi0sZ/8QbVNVumzMeM876+oDtcqV7XU1lCfM5ayxH3fsg27Tld4qZLXL3kibni7AozAFudveXEtPPA1e7e6Pf+G1VFIbf/AKlZRXb3d6vuIgF13wza5aZrfa4di41Id47/YqLAJpcL2HXXUPztVTd8ttquY293iiXWddyaMlzh10/Db3PG3tsc3I1laWpz1ljOVr8GXWH6mrYsgUqK9v87+x/vk9VxWYKq6v46+7DeHL/Y1qcN2xRiHLsI0tzW9p3h5ldBFwEMHDgwDZ58lUNvDkauj2qZWarsB/kxl576n1h111br8u65bXVcjPxv26LZbakIKQ+d5gMLXn/NydXU1nS3l5dDV98AWvWMODNf3JA+Xp2rtjATuXrKd78JZ2rK4nXVNO5pgpeu7dlX95trbAwuHTuzKFbnMpYnMpYAe/0Gdz4a20jURaFlcCAlOn+wKp0M7r7NGAaBC2FtnjyXYuL0v5q2DXxqyxblpmtwv7C27WRX7mp6yXsumtovpaqW35bLbex1xvlMlvaUmjO+mnofd7YY5uTq275/bp3pqq0lIFln9Jvwxp2Ll9Pz/L19K8ph9HTYc0a+Pzz4O+6dcGveeCxesv7srCIzQWFVHUqoLawEGo/hc6dk1/IdO4M3btvnU69vf5tbfU3Hgfb+nv5tAZaN5n8TomyKMwGLjOzxwg6mNc31Z/QlsaP3Cvttrq6Tq5sWWa2GnvogFB9CnWvvan1EnbdpZuvpeIxSy6/LZaburxMZIyiTyHs+mnsfT5+5F7N7lMoqK2m3+elDF63ioFlnzJ4w2cc02UTPPQD/rrifWKV2/5Cr+oUo6bnzlDRB3r3hgMOCP727g29ekHv3ry0Dm5e+DmfdelOWdGOVMXiyRxTTj+QflnYpxDFd0rGioKZzQBGAL3MbCUwAYgDuPu9wFzgRGA5UA6cl6ks6dR10rRlr34mlpmtfn5q0IEcdu8jaHy9hF139edrq72PUpfbFnsf1c/YFnsfpVs/Da2LdHsflezWs8V7HzW03sPsfVR3e0N7/RzWs4DnHnqGXd57m6FlKzm0fBU7fPAesarK5DKqdtiR+JCvwr77Ejv5ZF6P9+ShTzrxZqwH8V37cvGoYZx6cP+0z1/nG8CFib2PqnJk76MovlMy2tGcCW3V0SwiEaithaVL4eWX4ZVXgr/vvJPcxMOAAbD//sFlv/1g771h991hp5222awizZcLHc0i0hFs3Ahz5sBTT8H8+UHHLwSbdQ47DMaODf6WlEBxcbRZRUVBRDJg06agEMycCXPnwubNsMsuMGoUjBgBw4cHLQD9+s86Kgoi0jbKy4MCMHNmUBAqKqBvX7jwQjjrLDjiCOik4daynYqCiLTc5s3wpz8FheCZZ4IWQp8+cO65QSH4xjcgFos6pTSDioKINN+//w133gmPPhr0GfTqBePGwZlnwn/8BxToqyVX6T8nIuEtXgwTJgStgsLCoJP4nHPg6KNVCPKE/osi0rSVK+G66+CRR4LdQ2+4AS69FL7ylaiTSRtTURCRhm3cCLfcArfdFhxLMH48XHutdh3NYyoKIrK96mqYPj3YVLR6NZx9Ntx4IwwaFHUyyTAVBRHZyh3++Ef4yU+CI42PPDLYvfTrX486mbQT7TQsIoHSUjjhBDj5ZKipgVmz4K9/VUHoYNRSEBF4/HG45JLg/AB33hl0IsfjUaeSCKilINKRlZUFu5SOGQN77gmvvw7/9V8qCB2YioJIR7V0aTAI3cyZMGkSvPQSDBkSdSqJmDYfiXREM2fCeedBjx5Bv8ERR0SdSLKEWgoiHUlNDVx9dTAu0UEHwaJFKgiyDbUURDqKykr49reDVsIllwQdyoWFUaeSLKOiINIRbN4MZ5wRHHMwZQpcdVXUiSRLqSiI5LsvvwxObjN/Ptx7L3z/+1EnkiymoiCSz8rK4MQT4Z//hIcfDoa3FmmEioJIvlqzBo47Dt56C554AkaPjjqR5AAVBZF8VFoKxx4LH3wQnPtg5MioE0mOUFEQyTerV8OIEfDZZ/DnPwdnQhMJSUVBJJ9s2gQnnRS0FJ5/Hg4/POpEkmNUFETyRXV1cHrMRYvg979XQZAWUVEQyQfu8KMfBf0H99wDp5wSdSLJURrmQiQf3HIL/PrXwclxLr006jSSw1QURHLdY48F500eMwYmT446jeQ4FQWRXLZkCZx/fnDazAcfhE76SEvr6B0kkqs2bIDTTw+Gv545Ezp3jjqR5IGMFgUzO97MlpnZcjO7Js39A81svpktNrM3zezETOYRyRvucOGF8N57weajvn2jTiR5ImNFwcxiwD3ACcA+wFgz26febNcDM919KDAG+FWm8ojklbvvDoauuOkmOOqoqNNIHslkS+EQYLm7r3D3SuAxYFS9eRzonrjeA1iVwTwi+eEf/4ArrwxGPh0/Puo0kmcyWRT6AR+nTK9M3JZqIjDOzFYCc4EfpluQmV1kZgvNbOGaNWsykVUkN3z+OZx5JgwcGHQsm0WdSPJMJotCuner15seCzzo7v2BE4FHzGy7TO4+zd1L3L2kd+/eGYgqkgPcg3MhrF4NTz4JxcVRJ5I8lMkjmlcCA1Km+7P95qELgOMB3P0fZtYF6AWszmAukdw0YwY8/XRwoNrQoVGnkTyVyZbCq8AQMxtsZoUEHcmz683zEfCfAGb2NaALoO1DIvWtWgU/+EEwntGVV0adRvJYxoqCu1cDlwHzgHcI9jJ6y8wmmVndwCxXAt8zszeAGcC57l5/E5NIx+YO3/sebNkCDz0EsVjUiSSPZXRAPHefS9CBnHrbDSnX3waGZzKDSM574AGYOxfuuguGDIk6jeQ5HdEsks0++AAuvxyOPjrYfCSSYSoKItmqtjYY1wiC1oLGNZJ2oPMpiGSrhx6C+fPhN7+BQYOiTiMdhH56iGSjtWuDo5WHDw/GOBJpJyoKItno6quhrCw4cY42G0k70rtNJNssWAD33w8//jHsv3/UaaSDUVEQySZVVXDxxTBgANxwQ9Pzi7QxdTSLZJOpU2HpUpg1C3bYIeo00gGppSCSLT79FH76UzjppGBYbJEIqCiIZIsJE2DzZrj99qiTSAemoiCSDZYsgenTg6OWNZSFREhFQSQbXHUV9OihzmWJnDqaRaL25z/D//1fsNmoZ8+o00gHp5aCSJSqq4PzI3z1qxrwTrKCWgoiUZo+Hd5+G556CgoLo04jopaCSGQ2bAj6EI48EkaPjjqNCKCWgkh0Jk+GNWuCE+iYRZ1GBFBLQSQaH34Id9wB48ZBSUnUaUSSVBREonD99UHr4Kabok4isg0VBZH2tmQJ/O538MMfBgPfiWQRFQWR9vY//wM77hicM0Eky6goiLSnV16BP/whOKvazjtHnUZkOyoKIu3phhugVy+4/PKok4ikpaIg0l7+/vdgOIvx43WuBMlaKgoi7WXiROjdW8NZSFbTwWsi7WHBAnj2WZgyBbp1izqNSIPUUhBpDxMnQp8+cMklUScRaZRaCiKZ9tJL8Nxz8ItfqJUgWU8tBZFMUytBckijLQUzWwJ4Q/e7+wFNPP54YCoQA6a7+81p5jkTmJh4njfc/eymY4vkiL/9DZ5/Hm67Dbp2jTqNSJOa2nx0UuJv3e4SjyT+ngOUN/ZAM4sB9wDHAiuBV81stru/nTLPEOBaYLi7rzOzPs3ML5LdJk6Er3wFLr446iQioTRaFNz9QwAzG+7uw1PuusbMFgCTGnn4IcByd1+RWMZjwCjg7ZR5vgfc4+7rEs+3uvkvQSRLvfgivPBCcJpNtRIkR4TtU+hmZt+omzCzI4Cmesz6AR+nTK9M3JZqT2BPM1tgZi8nNjeJ5IeJE6FvX7USJKeE3fvoAuABM+uRmC4Dzm/iMenOGlK/f6IAGAKMAPoDfzOz/dy9bJsFmV0EXAQwcODAkJFFIrRgAcyfH5wzoago6jQioYUqCu6+CDjQzLoD5u7rQzxsJZA6LnB/YFWaeV529yrgfTNbRlAkXq33/NOAaQAlJSUNdnyLZI3Jk4MB7773vaiTiDRLU3sf/biB2wFw99sbefirwBAzGwyUAmOA+nsWzQLGAg+aWS+CzUkrQiUXyVZvvgl//CP87Gc6LkFyTlMthR1bumB3rzazy4B5BLukPuDub5nZJGChu89O3Hecmb0N1ADj3X1tS59TJCvcfHMw4J3GOJIcZO65tTWmpKTEFy5cGHUMkfSWL4e99oIrr4Rbb406jUiSmS1y9yZPCB5q7yMz629mvzez1Wb2mZk9ZWb9Wx9TJM9MmQLxOFxxRdRJRFok7C6pvwVmA7sS7Fb6TOI2EamzahU8+CCcdx7sskvUaURaJGxR6O3uv3X36sTlQaB3BnOJ5J477oDq6uAkOiI5KmxR+NzMxplZLHEZB6hDWKTOF1/AvffCmDGw++5RpxFpsbBF4XzgTOBT4BPgdJo+eE2k4/jlL+HLL+Gaa6JOItIqYQ9e+wg4JcNZRHLTpk1w111w8smw//5RpxFplVBFwcx6EwxeNyj1Me6u1oLIfffB2rVw7bVRJxFptbBjH/0B+BvwHMFBZiICUFkZnFHtqKPg8MOjTiPSamGLQld3vzqjSURy0SOPQGkpPPBA1ElE2kTYjuY5ZnZiRpOI5JqaGrjlFjj4YDj22KjTiLSJpgbE20gw3LUB/21mlUBlYtrdvXvmI4pkqaefhnffhSeeAEs3UrxI7mnqzGstHhBPJK+5B8Nj77knjB4ddRqRNhN27CNLHLz2P4npAWZ2SGajiWSxefNg8eLguIRYLOo0Im0mbJ/Cr4DD2Xo+hC+BezKSSCQXTJ4M/fvDOedEnUSkTYXd++hQdz/YzBYDuPs6MyvMYC6R7PX3v8OLL8Kdd0KhPgaSX8K2FKrMLEbiHMuJg9lqM5ZKJJvVnWrzwgujTiLS5sIWhbuA3wN9zOxG4CXgpoylEslWb74Jc+bA5ZfrVJuSl8KOffQ7M1sE/CfB7qinuvs7GU0mko10qk3Jc2H3PtoDeN/d7wGWAseaWXFGk4lkm/feg8cfh0sugZ12ijqNSEaE3Xz0FFBjZl8FpgODgf/NWCqRbKRTbUoHELYo1Lp7NXAaMNXdrwB0vkHpOD75BH77W51qU/Jec/Y+Ggt8B5iTuC2emUgiWei224KxjnSqTclzYYvCeQQHr93o7u+b2WDg0czFEskia9cGp9ocO1an2pS8F3bvo7eBH6VMvw/cnKlQIlll6tTg7Go6iY50AGHPvPY+iQPXUrm7fjZJftuwAe6+Oxj0bp99ok4jknFhh7koSbneBTgD6Nn2cUSyzK9+BWVlcN11UScRaReh+hTcfW3KpdTd7wS+meFsItEqL4fbb4eRI2HYsKjTiLSLsJuPDk6Z7ETQctC5FiS/3X8/rFmjVoJ0KGE3H92Wcr0a+AA4s83TiGSLykq49VY48sjgItJBhN376OhMBxHJKo88AitXwvTpUScRaVdhxz7qYWa3m9nCxOU2M+sR4nHHm9kyM1tuZtc0Mt/pZuZmVtLQPCLtpro6GPhu2DA47rio04i0q7AHrz0AbCTYZHQmsAH4bWMPSJx/4R7gBGAfYKyZbbdPn5ntSHAMxCvhY4tk0BNPwPLlQV+CWdRpRNpV2KKwh7tPcPcVictPgaaOUTgEWJ6YvxJ4DBiVZr6fAbcCm0OnFsmU2lq46abgmIRR6d6uIvktbFGoMLNv1E2Y2XCgoonH9AM+TplembgtycyGAgPcfQ6NMLOL6jZdrVmzJmRkkRZ45hlYujQ4erlT2I+HSP4Iu/fRJcBDKf0I64DvNvGYdO3u5FHRZtYJuAM4t6knd/dpwDSAkpKS7Y6sFmkT7nDjjcH4RmPGRJ1GJBJhi8I7BJt49gCKgfXAqcCbjTxmJTAgZbo/sCplekdgP+AvFmy37QvMNrNT3H1hyFwibWfuXHj1VbjvPigI+9EQyS9h3/l/AMqA14DSkI95FRiSGFG1FBgDnF13p7uvB3rVTZvZX4CrVBAkEu4wcSIMHgzfbaoRLJK/whaF/u5+fHMW7O7VZnYZMA+IAQ+4+1tmNglY6O6zm5lVJHPmzIGFC4OjmOM6VYh0XObe9CZ6M5sG3O3uSzIfqXElJSW+cKEaE9KG3KGkJBj47l//UlGQvGRmi9y9yWPBGm0pmNkSgs7hAuA8M1sBbCHoRHZ3P6AtwopEavZseO01ePBBFQTp8JrafHRSu6QQiUptLUyYAEOGwDnnRJ1GJHKNFgV3/7C9gohEYtYseOMNePhh7XEkQviD10TyT21tsMfRnnsG518WkdB7H4nkn6efhiVL4NFH1UoQSVBLQTqm6mq4/nrYe28dvSySQj+PpGN66CFYtixoLcRiUacRyRpqKUjHU1ER7HF02GFw6qlRpxHJKmopSMfzy19CaSn87nc6X4JIPWopSMdSVgaTJ8MJJ8BRR0WdRiTrqChIx3LLLVsLg4hsR0VBOo5Vq2DqVDj7bDjwwKjTiGQlFQXpOK6/PtgVddKkqJOIZC0VBekYFi0KBry7/PLgzGoikpaKguQ/d7jiCujVC667Luo0IllNu6RK/nvySfjb3+A3v4EePZqeX6QDU0tB8tvmzTB+PBxwAFxwQdRpRLKeWgqS3+64Az78EF54QcNZiISgloLkr1Wr4KabgqEsjj466jQiOUFFQfLXFVdAVRX84hdRJxHJGSoKkp/mzYOZM4O9jfbYI+o0IjlDRUHyT0UF/OAHwRnVfvKTqNOI5BR1NEv+mTwZ3nsPnn8eOneOOo1ITlFLQfLLsmXBoHfnnAPf/GbUaURyjoqC5A93uPRSKCqC226LOo1ITtLmI8kf06YFxyPcey985StRpxHJSWopSH54/3248ko45hi46KKo04jkLBUFyX21tcEQFp06wfTpOsWmSCto85Hkvl//GubPh/vug912izqNSE7LaEvBzI43s2VmttzMrklz/4/N7G0ze9PMnjczfaKleVasCI5FGDlSA96JtIGMFQUziwH3ACcA+wBjzWyferMtBkrc/QDgSeDWTOWRPFRdDd/5DsTj2mwk0kYy2VI4BFju7ivcvRJ4DBiVOoO7z3f38sTky0D/DOaRfDNhAixYEGw+6q+3jkhbyGRR6Ad8nDK9MnFbQy4A/pTBPJJPnn02OHL5ggtg7Nio04jkjUx2NKdry3vaGc3GASXAUQ3cfxFwEcDAgQPbKp/kqk8/hXHj4Gtfg7vuijqNSF7JZEthJTAgZbo/sKr+TGZ2DHAdcIq7b0m3IHef5u4l7l7Su3fvjISVHFFbC9/+NmzcGIyC2rVr1IlE8komi8KrwBAzG2xmhcAYYHbqDGY2FPgNQUFYncEski8mTYLnngtaCPvuG3UakbyTsaLg7tXAZcA84B1gpru/ZWaTzOyUxGxTgB2AJ8zsdTOb3cDiROCpp+CnP4Xvfle7n4pkiLmn3cyftUpKSnzhwoVRx5D29sYbcMQRcMABwYFqXbpEnUgkp5jZIncvaWo+DXMh2W/1ajjlFOjZE37/exUEkQzSMBeS3Sor4VvfCgrDSy9B375RJxLJayoKkr1qaoI9jV56CWbMgGHDok4kkve0+Uiykztcdlmw2+mUKTBmTNSJRDoEFQXJThMmBCfLufpquOqqqNOIdBgqCpJ9pk6Fn/0s2O108uSo04h0KCoKkl3uuQcuvxxGjw5aChr5VKRdqShI9rjzzqAfYdSooGO5QPtBiLQ3FQXJDrfeCldcEex++sQT0Llz1IlEOiQVBYlWbW3QkXz11cEeRjNmBCfNEZFIqH0u0dm8OehM/t//DTYbTZ0KnfQ7RSRKKgoSjU8/DTqTX34ZbroJrrlGncoiWUBFQdrf4sVBZ/LatcHIp6edFnUiEUlQW13ajztMmwaHHx5cf+klFQSRLKOiIO1j48ZgHKPvfx9GjIDXXoOhQ6NOJSL1qChI5v31r8F5EGbMgJ//HObOBZ1WVSQrqShI5mzaBD/+MRx9NMRi8OKLcN112sNIJIvp0yltzx2efhq+9jW44w64+GJ4/XUYPjzqZCLSBBUFaVvLl8OJJwZHJhcXB62DX/0Kdtgh6mQiEoKKgrSNTz4JDkDbZx9YsCBoIbz2Ghx5ZNTJRKQZdJyCtM4XXwTjFt11V3DqzAsvhBtugF13jTqZiLSAioK0zPvvB6Oa3n8/lJfD2WfDxInw1a9GnUxEWkFFQcJzDzYN3X03PPlksBfR2WfD+PGw335RpxORNqCiIE377DN4+OGgVbBsGXTvHoxs+sMfQv/+UacTkTakoiDpffEFzJoFM2fC889DdXWwS+k118AZZ0C3blEnFJEMUFGQgDssXQrPPgvz5sELLwSFYPfd4cor4dxzYe+9o04pIhmmotCRffIJPPdcUAieey6YBthrr+BI5DPPhIMP1pDWIh2IikJHUVYWHDewaNHWv+++G9y3885wzDFw3HFw7LEwYEC0WUUkMioK+aa2FkpL4V//2vrlv2gRrFixdZ6BA4MWwPnnB4XgoIM0HpGIACoKuccd1q2DDz+Ejz7a9u+77waX8vKt8w8eDMOGBQeVDRsWDFetEUpFpAEZLQpmdjwwFYgB09395nr3dwYeBoYBa4Gz3P2DTGSZtbiUKfOWsaqsgl2Lixg/ci9OHdqvWcu4ftYSZrzyMTXuxMw4bPed+GBtxTbLBJg4+y3KKqoA2KlrnAkn75t8rlmLS7e5H3d2idcyao8d+cdrK2D9enpWbKBv+TqKN62n16Yydi4vo9emMvp8uY5+G9fQrbJim1xbCgop7d6bD4v7smKfY1nRsx8revbn7T6Dqe5RzI2j99/mtdZ/HWMPHUDJbj23zZUHCmNGZY1vd/tOXeOsK68iZkaNb73fgK6FMTZV1iTvKy6KU1ldQ3lVbfKxdf/P+v/LTga1Dv1a+P6qe4+WllUkn7/ub0uXKdJc5r79h6ZNFmwWA/4NHAusBF4Fxrr72ynzXAoc4O4Xm9kYYLS7n9XYcktKSnzhwoXNyjJrcSnXPr2Eiqqa5G1F8RiTT9u/8Q+ZO9TUQFUVk2a9wVMvf0C8tpp4TTVdqivpUr2FLlWVdKmupHN1Jd1qKyms2kJh4rYu1VvYYUsFPSo3MaJvZ+JfbuSj9z+h25ZydtyyiR0Tf+O1NQ1G+LKwiLVde/B512I+71ZMafc+lPboQ2n33pR278Oq7r1Z27VHo53BsU7GbWccyKlD+3H9rCU8+vJH281jQGbeCfknHjPO+voAHv/nx1TVpl9rod5fKdK9R1u7TJFUZrbI3UuanC+DReFwYKK7j0xMXwvg7pNT5pmXmOcfZlYAfAr09kZCtaQoDL/5BUrLKrj0HzM57t2XKaitoaCmmiKrZbcd41BVFVyqq7der7u0Uo11YmPnrlR06caXXbrxRUERGzt3ZWPnbnzZuWvy+sbO3diQuL6uqDufdyvm86492Bzv0uoMEPx6XXDNN9nj2rnb/DqWlqnfykinbp2HUfcebUpzlimSKmxRyOTmo37AxynTK4FDG5rH3avNbD2wM/B56kxmdhFwEcDAgQObHWRV4sO2JRZnfZcdqOoUozpWQHWnAnYbNhDi8eBSULD1euqloIAb/2851bEYVZ0KqIoVsLmgkM0FndlSUMiWgsJgOl53W5zNBZ3ZXFBIebwLmFH3Oz6qr+O6daCC0DbCrMdVIb7kmztvc5Yp0hKZLArptmfU/ySFmQd3nwZMg6Cl0NwguxYXUVpWwf2HjOb+Q0Ynb+9XXMRJIX91PbChdb+wdy0uAgj1azAT6p4/zC9caVqY9Vi3zsOoe4+GmU8kkzK5H+JKIHWH9/7AqobmSWw+6gF80dZBxo/ci6J4bJvbiuKxZMdwGGMPbXrf/Xgno1OaMhePGeNH7sX4kXsRTzdDhsU6WfK1NvQ6dHhaePFY0Dnf2P+yue+vdO/R1i5TpCUyWRReBYaY2WAzKwTGALPrzTMb+G7i+unAC431J7TUqUP7Mfm0/elXXIQRtBCa22H381P3Z9xhA4klOnRjZgzfo+c2y5xyxoHcfuZBFBfFk4/bqWucKacHnbynDu3HlDMO3Ob+unnGHTZwm9vjDfw3zwQbAAAFZUlEQVRnusY7bXNfJ4Mhfbo1+KXerTCW7GRu6HWMO2wgd5x10Ha5cl1hLP1a2alr8Dpj9TrnjWB9pd5XXBSna8oKr/t//vzU/bf7X9bViJa8v1Lfo6nPX/e3JcsUaYmMdTQDmNmJwJ0Eu6Q+4O43mtkkYKG7zzazLsAjwFCCFsIYd1/R8BJb1tEsItLRZUNHM+4+F5hb77YbUq5vBs7IZAYREQlPYxuIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgkqSiIiEhSRg9eywQzWwN8GHWOFL2oN4BfDlH2aORydsjt/B05+27u3uQZtnKuKGQbM1sY5ijBbKTs0cjl7JDb+ZW9adp8JCIiSSoKIiKSpKLQetOiDtAKyh6NXM4OuZ1f2ZugPgUREUlSS0FERJJUFEIys+PNbJmZLTeza9Lcf66ZrTGz1xOXC6PIWZ+ZPWBmq81saQP3m5ndlXhdb5rZwe2dsSEhso8ws/Up6/yGdPNFwcwGmNl8M3vHzN4ys/9KM09WrvuQ2bN53Xcxs3+a2RuJ/D9NM09nM3s8se5fMbNB7Z90eyGzZ/a7xt11aeJCcJKg94DdgULgDWCfevOcC/wy6qxpsv8HcDCwtIH7TwT+RHDiscOAV6LO3IzsI4A5UedsINsuwMGJ6zsC/07znsnKdR8yezavewN2SFyPA68Ah9Wb51Lg3sT1McDjUeduRvaMfteopRDOIcByd1/h7pXAY8CoiDOF4u4v0vh5r0cBD3vgZaDYzHZpn3SNC5E9a7n7J+7+WuL6RuAdoP65NLNy3YfMnrUS6/PLxGQ8canfeToKeChx/UngP80s8lOVh8yeUSoK4fQDPk6ZXkn6D8m3EpsBnjSzAe0TrdXCvrZsdXiiqf0nM9s36jDpJDZNDCX41Zcq69d9I9khi9e9mcXM7HVgNfCsuze47t29GlgP7Ny+KdMLkR0y+F2johBOul8Q9av3M8Agdz8AeI6tv0KyXZjXlq1eIzh0/0DgbmBWxHm2Y2Y7AE8Bl7v7hvp3p3lI1qz7JrJn9bp39xp3PwjoDxxiZvvVmyVr132I7Bn9rlFRCGclkFqN+wOrUmdw97XuviUxeR8wrJ2ytVaTry1bufuGuqa2B+cDj5tZr4hjJZlZnOBL9Xfu/nSaWbJ23TeVPdvXfR13LwP+Ahxf767kujezAqAHWbapsqHsmf6uUVEI51VgiJkNNrNCgo6p2akz1NsWfArBdthcMBv4TmJPmMOA9e7+SdShwjCzvnXbgc3sEIL389poUwUSue4H3nH32xuYLSvXfZjsWb7ue5tZceJ6EXAM8K96s80Gvpu4fjrwgid6caMUJnumv2sK2nJh+crdq83sMmAewZ5ID7j7W2Y2CVjo7rOBH5nZKUA1wS+OcyMLnMLMZhDsKdLLzFYCEwg6r3D3e4G5BHvBLAfKgfOiSbq9ENlPBy4xs2qgAhiTDR/shOHAt4Elie3DAP8NDISsX/dhsmfzut8FeMjMYgTFaqa7z6n3eb0feMTMlhN8XsdEF3cbYbJn9LtGRzSLiEiSNh+JiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCSAaY2QfZeDCXSFNUFEREJElFQaSVzGyWmS1KjH9/UdR5RFpDRzSLtN757v5FYliCV83sqagDibSUioJI6/3IzEYnrg8AhkQZRqQ1VBREWsHMRhAMWna4u5eb2V+ALpGGEmkF9SmItE4PYF2iIOxNcFpNkZyloiDSOn8GCszsTeBnwMsR5xFpFY2SKiIiSWopiIhIkoqCiIgkqSiIiEiSioKIiCSpKIiISJKKgoiIJKkoiIhIkoqCiIgk/T88gTJFdrKNdAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c307fea58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities again\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.64722323])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted log-odds for al=2 using the equation\n",
    "logodds = logreg.intercept_ + logreg.coef_[0] * 2\n",
    "logodds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.91022919])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to odds\n",
    "odds = np.exp(logodds)\n",
    "odds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.65638445])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert odds to probability\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.65638445])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=2 using the predict_proba method\n",
    "logreg.predict_proba(2)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['al'], array([4.18040386]))"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the coefficient for al\n",
    "feature_cols, logreg.coef_[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 4.18 unit increase in the log-odds of 'household'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9920580839167457"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) increases the log-odds by 4.18\n",
    "logodds = 0.64722323 + 4.1804038614510901\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.99205808])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=3 using the predict_proba method\n",
    "logreg.predict_proba(3)[:, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bottom line:** Positive coefficients increase the log-odds of the response (and thus increase the probability), and negative coefficients decrease the log-odds of the response (and thus decrease the probability)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-7.71358449])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the intercept\n",
    "logreg.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** For an 'al' value of 0, the log-odds of 'household' is -7.71."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.00044652])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to probability\n",
    "logodds = logreg.intercept_\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That makes sense from the plot above, because the probability of household=1 should be very low for such a low 'al' value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Logistic regression beta values](images/logistic_betas.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the $\\beta_0$ value shifts the curve **horizontally**, whereas changing the $\\beta_1$ value changes the **slope** of the curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing Logistic Regression with Other Models\n",
    "\n",
    "Advantages of logistic regression:\n",
    "\n",
    "- Highly interpretable (if you remember how)\n",
    "- Model training and prediction are fast\n",
    "- No tuning is required (excluding regularization)\n",
    "- Features don't need scaling\n",
    "- Can perform well with a small number of observations\n",
    "- Outputs well-calibrated predicted probabilities\n",
    "\n",
    "Disadvantages of logistic regression:\n",
    "\n",
    "- Presumes a linear relationship between the features and the log-odds of the response\n",
    "- Performance is (generally) not competitive with the best supervised learning methods\n",
    "- Can't automatically learn feature interactions"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
